Max level shown:
12025-12-23T22:00:01.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 476 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = f0bae49b-67ed-4d52-8e09-f4269f603f35 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
22025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
32025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
42025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
52025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
62025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
72025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
82025-12-23T22:00:04.619ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-23 21:50:04.619680819 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
92025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
102025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
112025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
122025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
132025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
142025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
152025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
162025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
172025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
182025-12-23T22:00:04.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
192025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
202025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
212025-12-23T22:00:04.620ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
222025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
232025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
242025-12-23T22:00:04.620ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = ExternalNetworking step id = Ensure
252025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
262025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
272025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
282025-12-23T22:00:04.620ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2bc6e36d-0fab-4746-9740-073e15922b00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 719959600 } })
292025-12-23T22:00:04.620ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a98c752e-5a58-41fa-9f53-a8d4f529a42f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e9c3ce58-81e4-4140-937f-1b169b78f235.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 720098384 } })
302025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
312025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
322025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
332025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
342025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
352025-12-23T22:00:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
362025-12-23T22:00:04.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
372025-12-23T22:00:04.621ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
382025-12-23T22:00:04.621ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.184472ms iteration = 11 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
392025-12-23T22:00:04.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
402025-12-23T22:00:04.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
412025-12-23T22:00:04.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
422025-12-23T22:00:04.622ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
432025-12-23T22:00:04.622ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.85697ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
442025-12-23T22:00:04.622ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
452025-12-23T22:00:04.623ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-23 21:50:04.619680819 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
462025-12-23T22:00:04.623ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.243029ms iteration = 11 status = {"expiration":"2025-12-23T21:50:04.619680819Z","pruned":[]}
472025-12-23T22:00:04.624ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 723999764 } })
482025-12-23T22:00:04.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
492025-12-23T22:00:04.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 724373034 } })
502025-12-23T22:00:04.624ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
512025-12-23T22:00:04.625ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 725054734 } })
522025-12-23T22:00:04.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
532025-12-23T22:00:04.629ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
542025-12-23T22:00:04.629ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
552025-12-23T22:00:04.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
562025-12-23T22:00:04.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
572025-12-23T22:00:04.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
582025-12-23T22:00:04.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
592025-12-23T22:00:04.637ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.979717ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
602025-12-23T22:00:04.637ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
612025-12-23T22:00:04.637ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
622025-12-23T22:00:04.637ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
632025-12-23T22:00:04.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
642025-12-23T22:00:04.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
652025-12-23T22:00:04.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
662025-12-23T22:00:04.639ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
672025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
682025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 15 reason = Timeout
692025-12-23T22:00:04.640ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
702025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
712025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
722025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 25 reason = Timeout
732025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
742025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
752025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
762025-12-23T22:00:04.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
772025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
782025-12-23T22:00:04.641ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
792025-12-23T22:00:04.641ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
802025-12-23T22:00:04.641ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
812025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
822025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
832025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
842025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
852025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
862025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
872025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
882025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
892025-12-23T22:00:04.641ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
902025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
912025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 35 reason = Timeout
922025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
932025-12-23T22:00:04.641ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
942025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
952025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.16\u{b5}s iteration = 35 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
962025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
972025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
982025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
992025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 11 reason = Timeout
1002025-12-23T22:00:04.641ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
1012025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
1022025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
1032025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
1042025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
1052025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1062025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 4 reason = Timeout
1072025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
1082025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
1092025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
1102025-12-23T22:00:04.641ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
1112025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
1122025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1132025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1142025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1152025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
1162025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
1172025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
1182025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
1192025-12-23T22:00:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1202025-12-23T22:00:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
1212025-12-23T22:00:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1222025-12-23T22:00:04.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1232025-12-23T22:00:04.648ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.351957ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
1242025-12-23T22:00:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1252025-12-23T22:00:04.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1262025-12-23T22:00:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1272025-12-23T22:00:04.651ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
1282025-12-23T22:00:04.651ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 10.211899ms iteration = 11 status = {"by_rx":{},"error":null}
1292025-12-23T22:00:04.653ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1302025-12-23T22:00:04.653ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1312025-12-23T22:00:04.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1322025-12-23T22:00:04.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1332025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1342025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
1352025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1372025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1392025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1402025-12-23T22:00:04.655ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1412025-12-23T22:00:04.656ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1422025-12-23T22:00:04.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1432025-12-23T22:00:04.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1442025-12-23T22:00:04.658ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1452025-12-23T22:00:04.659ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1462025-12-23T22:00:04.659ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1472025-12-23T22:00:04.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1482025-12-23T22:00:04.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1492025-12-23T22:00:04.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
1502025-12-23T22:00:04.660ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
1512025-12-23T22:00:04.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
1522025-12-23T22:00:04.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1532025-12-23T22:00:04.661ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1542025-12-23T22:00:04.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
1552025-12-23T22:00:04.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
1562025-12-23T22:00:04.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1572025-12-23T22:00:04.662ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 20.89442ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"215.0.0","globs_reprocessed":{},"no_receivers":[]}
1582025-12-23T22:00:04.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
1592025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
1602025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
1612025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
1622025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
1632025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
1642025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
1652025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
1662025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
1672025-12-23T22:00:04.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
1682025-12-23T22:00:04.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
1692025-12-23T22:00:04.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1702025-12-23T22:00:04.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1712025-12-23T22:00:04.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1722025-12-23T22:00:04.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1732025-12-23T22:00:04.667ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1742025-12-23T22:00:04.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1752025-12-23T22:00:04.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1762025-12-23T22:00:04.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1772025-12-23T22:00:04.669ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1782025-12-23T22:00:04.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1792025-12-23T22:00:04.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1802025-12-23T22:00:04.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1812025-12-23T22:00:04.670ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
1822025-12-23T22:00:04.670ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1832025-12-23T22:00:04.672ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
1842025-12-23T22:00:04.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
1852025-12-23T22:00:04.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
1862025-12-23T22:00:04.673ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
1872025-12-23T22:00:04.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
1882025-12-23T22:00:04.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
1892025-12-23T22:00:04.678ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
1902025-12-23T22:00:04.678ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
1912025-12-23T22:00:04.678ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1922025-12-23T22:00:04.678ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-23 21:50:07.200941 UTC dns_group = external generation = 2
1932025-12-23T22:00:04.678ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
1942025-12-23T22:00:04.678ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 58.368567ms iteration = 11 status = {"generation":2}
1952025-12-23T22:00:04.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
1962025-12-23T22:00:04.679ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
1972025-12-23T22:00:04.679ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
1982025-12-23T22:00:04.679ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
1992025-12-23T22:00:04.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
2002025-12-23T22:00:04.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
2012025-12-23T22:00:04.680ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
2022025-12-23T22:00:04.680ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
2032025-12-23T22:00:04.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
2042025-12-23T22:00:04.680ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
2052025-12-23T22:00:04.682ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2062025-12-23T22:00:04.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
2072025-12-23T22:00:04.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
2082025-12-23T22:00:04.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2092025-12-23T22:00:04.685ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2102025-12-23T22:00:04.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2112025-12-23T22:00:04.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
2122025-12-23T22:00:04.687ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
2132025-12-23T22:00:04.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
2142025-12-23T22:00:04.689ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2152025-12-23T22:00:04.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2162025-12-23T22:00:04.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
2172025-12-23T22:00:04.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
2182025-12-23T22:00:04.690ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
2192025-12-23T22:00:04.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
2202025-12-23T22:00:04.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
2212025-12-23T22:00:04.698ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
2222025-12-23T22:00:04.698ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2232025-12-23T22:00:04.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2242025-12-23T22:00:04.700ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2252025-12-23T22:00:04.704ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
2262025-12-23T22:00:04.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
2272025-12-23T22:00:04.708ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
2282025-12-23T22:00:04.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
2292025-12-23T22:00:04.711ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
2302025-12-23T22:00:04.715ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
2312025-12-23T22:00:04.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
2322025-12-23T22:00:04.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
2332025-12-23T22:00:04.715ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2342025-12-23T22:00:04.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
2352025-12-23T22:00:04.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2362025-12-23T22:00:04.721ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:153 status = TufRepoPrunerDetails { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
2372025-12-23T22:00:04.721ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
2382025-12-23T22:00:04.721ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 80.175663ms iteration = 4 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"status":"enabled","warnings":[]}
2392025-12-23T22:00:04.723ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2402025-12-23T22:00:04.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2412025-12-23T22:00:04.726ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
2422025-12-23T22:00:04.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
2432025-12-23T22:00:04.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
2442025-12-23T22:00:04.727ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
2452025-12-23T22:00:04.727ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
2462025-12-23T22:00:04.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
2472025-12-23T22:00:04.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2482025-12-23T22:00:04.733ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
2492025-12-23T22:00:04.736ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
2502025-12-23T22:00:04.738ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
2512025-12-23T22:00:04.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
2522025-12-23T22:00:04.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
2532025-12-23T22:00:04.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
2542025-12-23T22:00:04.749ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2552025-12-23T22:00:04.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2562025-12-23T22:00:04.754ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
2572025-12-23T22:00:04.754ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = Some(02039deb-aee8-405c-80d5-e584bd199fe1 (blueprint)) file = nexus/types/src/quiesce.rs:525 generation = 1
2582025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 113.255627ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
2592025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
2602025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
2612025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2622025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
2632025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2642025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
2652025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2662025-12-23T22:00:04.754ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2672025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
2682025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
2692025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2702025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
2712025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2722025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2732025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2742025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2752025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2762025-12-23T22:00:04.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2772025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
2782025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
2792025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
2802025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
2812025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2822025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
2832025-12-23T22:00:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2842025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 53 reason = Timeout
2852025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
2862025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
2872025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2882025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2892025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
2902025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2912025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 41 reason = Timeout
2922025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
2932025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
2942025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2952025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
2962025-12-23T22:00:04.757ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2972025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2982025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
2992025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3002025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
3012025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3022025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
3032025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3042025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 121 reason = Timeout
3052025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
3062025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 41 reason = Timeout
3072025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
3082025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
3092025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
3102025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
3112025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3122025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3132025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3142025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3152025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3162025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
3172025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
3182025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
3192025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3202025-12-23T22:00:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3212025-12-23T22:00:04.758ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
3222025-12-23T22:00:04.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
3232025-12-23T22:00:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
3242025-12-23T22:00:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3252025-12-23T22:00:04.759ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
3262025-12-23T22:00:04.760ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.791\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"sps":[]}
3272025-12-23T22:00:04.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3282025-12-23T22:00:04.761ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = SledList step id = Fetch
3292025-12-23T22:00:04.761ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3302025-12-23T22:00:04.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3312025-12-23T22:00:04.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3322025-12-23T22:00:04.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3332025-12-23T22:00:04.762ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3342025-12-23T22:00:04.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3352025-12-23T22:00:04.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3362025-12-23T22:00:04.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3372025-12-23T22:00:04.763ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3382025-12-23T22:00:04.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3392025-12-23T22:00:04.767ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.498863ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
3402025-12-23T22:00:04.767ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3412025-12-23T22:00:04.767ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3422025-12-23T22:00:04.767ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3432025-12-23T22:00:04.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3442025-12-23T22:00:04.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3452025-12-23T22:00:04.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3462025-12-23T22:00:04.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
3472025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
3482025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
3492025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
3502025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
3512025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
3522025-12-23T22:00:04.775ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
3532025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 135.463535ms iteration = 15 status = {"blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","inventory_collection_id":"f948b43c-60d4-4f42-abfd-49b154566105","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
3542025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3552025-12-23T22:00:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
3562025-12-23T22:00:04.776ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 19.171451ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
3572025-12-23T22:00:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3582025-12-23T22:00:04.778ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3592025-12-23T22:00:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
3602025-12-23T22:00:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3612025-12-23T22:00:04.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3622025-12-23T22:00:04.780ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3632025-12-23T22:00:04.780ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3642025-12-23T22:00:04.781ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3652025-12-23T22:00:04.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3662025-12-23T22:00:04.783ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3672025-12-23T22:00:04.783ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3682025-12-23T22:00:04.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3692025-12-23T22:00:04.783ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3702025-12-23T22:00:04.783ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 29.065731ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3712025-12-23T22:00:04.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3722025-12-23T22:00:04.785ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.523432ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3732025-12-23T22:00:04.785ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
3742025-12-23T22:00:04.785ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 29.005588ms iteration = 121 status = {"config_updated":false}
3752025-12-23T22:00:04.786ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3762025-12-23T22:00:04.786ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = f948b43c-60d4-4f42-abfd-49b154566105 original_time_started = 2025-12-23 21:59:06.140319 UTC
3772025-12-23T22:00:04.786ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 29.697798ms iteration = 53 status = {"Loaded":{"collection_id":"f948b43c-60d4-4f42-abfd-49b154566105","time_loaded":"2025-12-23T22:00:04.756606824Z","time_started":"2025-12-23T21:59:06.140319Z"}}
3782025-12-23T22:00:04.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3792025-12-23T22:00:04.786ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3802025-12-23T22:00:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3812025-12-23T22:00:04.787ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3822025-12-23T22:00:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3832025-12-23T22:00:04.787ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3842025-12-23T22:00:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3852025-12-23T22:00:04.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3862025-12-23T22:00:04.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
3872025-12-23T22:00:04.788ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
3882025-12-23T22:00:04.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
3892025-12-23T22:00:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3902025-12-23T22:00:04.789ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3912025-12-23T22:00:04.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3922025-12-23T22:00:04.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3932025-12-23T22:00:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3942025-12-23T22:00:04.790ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3952025-12-23T22:00:04.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3962025-12-23T22:00:04.791ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
3972025-12-23T22:00:04.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3982025-12-23T22:00:04.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3992025-12-23T22:00:04.792ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4002025-12-23T22:00:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4012025-12-23T22:00:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4022025-12-23T22:00:04.797ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 897264595 } })
4032025-12-23T22:00:04.798ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
4042025-12-23T22:00:04.798ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1502, tv_nsec: 898410237 } })
4052025-12-23T22:00:04.799ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
4062025-12-23T22:00:04.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4072025-12-23T22:00:04.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4082025-12-23T22:00:04.800ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4092025-12-23T22:00:04.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4102025-12-23T22:00:04.802ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
4112025-12-23T22:00:04.802ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
4122025-12-23T22:00:04.802ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 45.284378ms iteration = 41 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
4132025-12-23T22:00:04.803ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 46.164103ms iteration = 41 status = "NoSitrep"
4142025-12-23T22:00:04.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4152025-12-23T22:00:04.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
4162025-12-23T22:00:04.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4172025-12-23T22:00:04.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4182025-12-23T22:00:04.814ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
4192025-12-23T22:00:04.814ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
4202025-12-23T22:00:04.814ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 173.249971ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
4212025-12-23T22:00:04.823ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 67.743372ms iteration = 21 status = 0
4222025-12-23T22:00:04.823ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4232025-12-23T22:00:04.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4242025-12-23T22:00:04.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4252025-12-23T22:00:04.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
4262025-12-23T22:00:04.824ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4272025-12-23T22:00:04.824ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
4282025-12-23T22:00:04.824ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 184.362804ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4292025-12-23T22:00:04.826ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4302025-12-23T22:00:04.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4312025-12-23T22:00:04.827ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4322025-12-23T22:00:04.827ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4332025-12-23T22:00:04.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4342025-12-23T22:00:04.829ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4352025-12-23T22:00:04.830ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
4362025-12-23T22:00:04.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4372025-12-23T22:00:04.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4382025-12-23T22:00:04.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4392025-12-23T22:00:04.830ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = DeployNexusRecords step id = Ensure
4402025-12-23T22:00:04.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4412025-12-23T22:00:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4422025-12-23T22:00:04.832ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4432025-12-23T22:00:04.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4442025-12-23T22:00:04.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4452025-12-23T22:00:04.841ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4462025-12-23T22:00:04.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4472025-12-23T22:00:04.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4482025-12-23T22:00:04.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4492025-12-23T22:00:04.845ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
4502025-12-23T22:00:04.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4512025-12-23T22:00:04.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4522025-12-23T22:00:04.846ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4532025-12-23T22:00:04.846ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4542025-12-23T22:00:04.846ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4552025-12-23T22:00:04.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
4562025-12-23T22:00:04.846ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4572025-12-23T22:00:04.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4582025-12-23T22:00:04.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
4592025-12-23T22:00:04.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
4602025-12-23T22:00:04.850ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
4612025-12-23T22:00:04.850ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4622025-12-23T22:00:04.851ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f31ff2e-96b6-499b-a328-834aa4762eb4", "content-length": "26", "date": "Tue, 23 Dec 2025 22:00:04 GMT"} })
4632025-12-23T22:00:04.851ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4642025-12-23T22:00:04.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4652025-12-23T22:00:04.851ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4662025-12-23T22:00:04.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4672025-12-23T22:00:04.852ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4682025-12-23T22:00:04.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4692025-12-23T22:00:04.855ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4702025-12-23T22:00:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
4712025-12-23T22:00:04.862ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4722025-12-23T22:00:04.862ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4732025-12-23T22:00:04.862ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4742025-12-23T22:00:04.862ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4752025-12-23T22:00:04.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4762025-12-23T22:00:04.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4772025-12-23T22:00:04.868ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4782025-12-23T22:00:04.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4792025-12-23T22:00:04.872ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4802025-12-23T22:00:04.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4812025-12-23T22:00:04.884ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4822025-12-23T22:00:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4832025-12-23T22:00:04.898ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4842025-12-23T22:00:04.898ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4852025-12-23T22:00:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
4862025-12-23T22:00:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4872025-12-23T22:00:04.902ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 145.647211ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4882025-12-23T22:00:04.902ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4892025-12-23T22:00:04.903ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4902025-12-23T22:00:04.903ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4912025-12-23T22:00:04.903ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4922025-12-23T22:00:04.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4932025-12-23T22:00:04.903ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4942025-12-23T22:00:04.904ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4952025-12-23T22:00:04.904ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1503, tv_nsec: 4480321 } })
4962025-12-23T22:00:04.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4972025-12-23T22:00:04.905ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4982025-12-23T22:00:04.906ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4992025-12-23T22:00:04.906ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
5002025-12-23T22:00:04.906ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 151.927418ms iteration = 61 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
5012025-12-23T22:00:04.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5022025-12-23T22:00:04.907ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5032025-12-23T22:00:04.907ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5042025-12-23T22:00:04.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
5052025-12-23T22:00:04.910ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5062025-12-23T22:00:04.910ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5072025-12-23T22:00:04.910ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5082025-12-23T22:00:04.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
5092025-12-23T22:00:04.910ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5102025-12-23T22:00:04.912ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5112025-12-23T22:00:04.912ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5122025-12-23T22:00:04.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5132025-12-23T22:00:04.913ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5142025-12-23T22:00:04.916ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5152025-12-23T22:00:04.916ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 160.074948ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5162025-12-23T22:00:04.918ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5172025-12-23T22:00:04.918ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5182025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5192025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
5202025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5212025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
5222025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5232025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-23 21:50:06.190854 UTC dns_group = internal generation = 1
5242025-12-23T22:00:04.919ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 300.376879ms iteration = 11 status = {"generation":1}
5252025-12-23T22:00:04.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5262025-12-23T22:00:04.925ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5272025-12-23T22:00:04.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5282025-12-23T22:00:04.928ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5292025-12-23T22:00:04.928ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5302025-12-23T22:00:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5312025-12-23T22:00:04.933ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5322025-12-23T22:00:04.933ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5332025-12-23T22:00:04.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5342025-12-23T22:00:04.934ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 177.609623ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
5352025-12-23T22:00:04.948ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 191.948359ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
5362025-12-23T22:00:04.964ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5372025-12-23T22:00:04.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5382025-12-23T22:00:04.967ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5392025-12-23T22:00:04.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5402025-12-23T22:00:04.971ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5412025-12-23T22:00:04.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5422025-12-23T22:00:04.975ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
5432025-12-23T22:00:04.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
5442025-12-23T22:00:04.976ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-23T21:50:00.171530Z, time_modified: 2025-12-23T21:50:00.171530Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }) }
5452025-12-23T22:00:04.976ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 356.416361ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]}},"default_endpoint":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5462025-12-23T22:00:04.980ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5472025-12-23T22:00:04.980ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
5482025-12-23T22:00:04.980ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 223.789928ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
5492025-12-23T22:00:04.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5502025-12-23T22:00:04.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5512025-12-23T22:00:04.985ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
5522025-12-23T22:00:04.985ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
5532025-12-23T22:00:04.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
5542025-12-23T22:00:04.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5552025-12-23T22:00:04.991ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5562025-12-23T22:00:04.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5572025-12-23T22:00:04.992ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = SledAgent step id = Ensure
5582025-12-23T22:00:04.994ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
5592025-12-23T22:00:04.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5602025-12-23T22:00:04.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5612025-12-23T22:00:04.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5622025-12-23T22:00:05.003ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
5632025-12-23T22:00:05.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5642025-12-23T22:00:05.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5652025-12-23T22:00:05.008ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5662025-12-23T22:00:05.008ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5672025-12-23T22:00:05.008ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5682025-12-23T22:00:05.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
5692025-12-23T22:00:05.042ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 285.50005ms iteration = 33 status = {"physical_disks_added":0}
5702025-12-23T22:00:05.043ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5712025-12-23T22:00:05.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5722025-12-23T22:00:05.050ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5732025-12-23T22:00:05.057ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
5742025-12-23T22:00:05.057ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5752025-12-23T22:00:05.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5762025-12-23T22:00:05.062ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
5772025-12-23T22:00:05.066ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5782025-12-23T22:00:05.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5792025-12-23T22:00:05.068ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5802025-12-23T22:00:05.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5812025-12-23T22:00:05.071ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5822025-12-23T22:00:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5832025-12-23T22:00:05.075ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5842025-12-23T22:00:05.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5852025-12-23T22:00:05.076ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 456.257289ms iteration = 21 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
5862025-12-23T22:00:05.078ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5872025-12-23T22:00:05.078ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5882025-12-23T22:00:05.078ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 323.07394ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5892025-12-23T22:00:05.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5902025-12-23T22:00:05.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5912025-12-23T22:00:05.081ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5922025-12-23T22:00:05.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5932025-12-23T22:00:05.088ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5942025-12-23T22:00:05.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5952025-12-23T22:00:05.091ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5962025-12-23T22:00:05.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5972025-12-23T22:00:05.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5982025-12-23T22:00:05.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5992025-12-23T22:00:05.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6002025-12-23T22:00:05.100ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6012025-12-23T22:00:05.100ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6022025-12-23T22:00:05.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6032025-12-23T22:00:05.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6042025-12-23T22:00:05.102ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6052025-12-23T22:00:05.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6062025-12-23T22:00:05.105ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6072025-12-23T22:00:05.105ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6082025-12-23T22:00:05.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6092025-12-23T22:00:05.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6102025-12-23T22:00:05.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6112025-12-23T22:00:05.108ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6122025-12-23T22:00:05.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6132025-12-23T22:00:05.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6142025-12-23T22:00:05.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6152025-12-23T22:00:05.116ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
6162025-12-23T22:00:05.116ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6172025-12-23T22:00:05.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6182025-12-23T22:00:05.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
6192025-12-23T22:00:05.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6202025-12-23T22:00:05.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6212025-12-23T22:00:05.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6222025-12-23T22:00:05.126ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
6232025-12-23T22:00:05.126ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6242025-12-23T22:00:05.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6252025-12-23T22:00:05.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6262025-12-23T22:00:05.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6272025-12-23T22:00:05.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6282025-12-23T22:00:05.137ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6292025-12-23T22:00:05.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6302025-12-23T22:00:05.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6312025-12-23T22:00:05.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6322025-12-23T22:00:05.143ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
6332025-12-23T22:00:05.143ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6342025-12-23T22:00:05.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6352025-12-23T22:00:05.146ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
6362025-12-23T22:00:05.146ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
6372025-12-23T22:00:05.146ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
6382025-12-23T22:00:05.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
6392025-12-23T22:00:05.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6402025-12-23T22:00:05.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6412025-12-23T22:00:05.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6422025-12-23T22:00:05.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6432025-12-23T22:00:05.153ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6442025-12-23T22:00:05.153ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6452025-12-23T22:00:05.154ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2fb7cad-0de3-4139-b814-c3b95e063b51", "content-length": "372", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
6462025-12-23T22:00:05.154ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6472025-12-23T22:00:05.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6482025-12-23T22:00:05.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6492025-12-23T22:00:05.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
6502025-12-23T22:00:05.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6512025-12-23T22:00:05.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6522025-12-23T22:00:05.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6532025-12-23T22:00:05.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6542025-12-23T22:00:05.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6552025-12-23T22:00:05.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
6562025-12-23T22:00:05.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6572025-12-23T22:00:05.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6582025-12-23T22:00:05.195ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
6592025-12-23T22:00:05.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6602025-12-23T22:00:05.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6612025-12-23T22:00:05.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6622025-12-23T22:00:05.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6632025-12-23T22:00:05.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6642025-12-23T22:00:05.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6652025-12-23T22:00:05.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6662025-12-23T22:00:05.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6672025-12-23T22:00:05.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6682025-12-23T22:00:05.225ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 605.625295ms iteration = 21 status = {}
6692025-12-23T22:00:05.228ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6702025-12-23T22:00:05.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6712025-12-23T22:00:05.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6722025-12-23T22:00:05.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6732025-12-23T22:00:05.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6742025-12-23T22:00:05.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6752025-12-23T22:00:05.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6762025-12-23T22:00:05.239ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
6772025-12-23T22:00:05.239ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6782025-12-23T22:00:05.239ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "184a75ca-2ec2-42f9-a798-64c4c442b56d", "content-length": "26", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
6792025-12-23T22:00:05.239ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6802025-12-23T22:00:05.241ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
6812025-12-23T22:00:05.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6822025-12-23T22:00:05.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6832025-12-23T22:00:05.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6842025-12-23T22:00:05.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6852025-12-23T22:00:05.252ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
6862025-12-23T22:00:05.252ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6872025-12-23T22:00:05.252ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), time_created: 2025-12-23T21:49:57.392343Z, time_modified: 2025-12-23T21:49:57.392343Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
6882025-12-23T22:00:05.252ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
6892025-12-23T22:00:05.252ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "19e9c2f9-aff1-494e-8855-726a4d475ba4", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
6902025-12-23T22:00:05.252ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6912025-12-23T22:00:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6922025-12-23T22:00:05.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932025-12-23T22:00:05.261ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6942025-12-23T22:00:05.261ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6952025-12-23T22:00:05.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6962025-12-23T22:00:05.262ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6734a6-d9da-4553-8cb4-db16ec78c0ea", "content-length": "691", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
6972025-12-23T22:00:05.262ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
6982025-12-23T22:00:05.262ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
6992025-12-23T22:00:05.262ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
7002025-12-23T22:00:05.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
7012025-12-23T22:00:05.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7022025-12-23T22:00:05.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7032025-12-23T22:00:05.264ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 507.836516ms iteration = 21 status = {}
7042025-12-23T22:00:05.265ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7052025-12-23T22:00:05.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7062025-12-23T22:00:05.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7072025-12-23T22:00:05.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7082025-12-23T22:00:05.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7092025-12-23T22:00:05.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7102025-12-23T22:00:05.269ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7112025-12-23T22:00:05.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b85fd2d6-de57-497b-a1c6-190373a83ab0", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7122025-12-23T22:00:05.270ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7132025-12-23T22:00:05.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142025-12-23T22:00:05.270ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7152025-12-23T22:00:05.270ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7162025-12-23T22:00:05.271ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7172025-12-23T22:00:05.271ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7b1c3e14-fbe8-4da1-8e2b-4b0f3caf237a", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
7182025-12-23T22:00:05.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7192025-12-23T22:00:05.273ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "808b85ed-f1f0-4527-bb70-eec20d5d9f50", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7202025-12-23T22:00:05.273ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7212025-12-23T22:00:05.273ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7222025-12-23T22:00:05.274ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 517.701414ms iteration = 31 status = {}
7232025-12-23T22:00:05.277ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
7242025-12-23T22:00:05.277ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 657.350496ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
7252025-12-23T22:00:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
7262025-12-23T22:00:05.277ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7272025-12-23T22:00:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
7282025-12-23T22:00:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7292025-12-23T22:00:05.278ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
7302025-12-23T22:00:05.279ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "f95be402-59be-4610-be45-5ff725fac40d", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
7312025-12-23T22:00:05.279ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
7322025-12-23T22:00:05.280ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7332025-12-23T22:00:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7342025-12-23T22:00:05.282ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
7352025-12-23T22:00:05.282ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 640.988984ms iteration = 11 status = {"errors":[],"probes_by_sled":{"ff85f534-787c-483e-ba89-a37cdea1dd38":0}}
7362025-12-23T22:00:05.282ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Dependency
7372025-12-23T22:00:05.282ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
7382025-12-23T22:00:05.282ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7392025-12-23T22:00:05.282ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7402025-12-23T22:00:05.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7412025-12-23T22:00:05.282ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
7422025-12-23T22:00:05.282ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 641.563341ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7432025-12-23T22:00:05.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7442025-12-23T22:00:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
7452025-12-23T22:00:05.287ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7462025-12-23T22:00:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7472025-12-23T22:00:05.288ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7482025-12-23T22:00:05.289ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7492025-12-23T22:00:05.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7502025-12-23T22:00:05.290ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7512025-12-23T22:00:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7522025-12-23T22:00:05.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7532025-12-23T22:00:05.292ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7542025-12-23T22:00:05.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7552025-12-23T22:00:05.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7562025-12-23T22:00:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7572025-12-23T22:00:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
7582025-12-23T22:00:05.297ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7592025-12-23T22:00:05.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
7602025-12-23T22:00:05.299ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
7612025-12-23T22:00:05.300ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "cf668a2a-ad44-4561-b54d-5350a6195829", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
7622025-12-23T22:00:05.300ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
7632025-12-23T22:00:05.300ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "27301b5a-41b2-4472-8ff6-5baa7d691efc", "content-length": "26", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
7642025-12-23T22:00:05.300ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
7652025-12-23T22:00:05.303ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 662.614795ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"ff85f534-787c-483e-ba89-a37cdea1dd38","time":"2025-12-23T21:50:05.297462872Z"},{"operation":"list","target_sled":"ff85f534-787c-483e-ba89-a37cdea1dd38","time":"2025-12-23T21:50:05.299781158Z"},{"generation":1,"operation":"put_config","target_sled":"ff85f534-787c-483e-ba89-a37cdea1dd38","time":"2025-12-23T21:55:05.152112546Z"},{"operation":"list","target_sled":"ff85f534-787c-483e-ba89-a37cdea1dd38","time":"2025-12-23T21:55:05.152454355Z"},{"generation":1,"operation":"put_config","target_sled":"ff85f534-787c-483e-ba89-a37cdea1dd38","time":"2025-12-23T22:00:05.300585609Z"},{"operation":"list","target_sled":"ff85f534-787c-483e-ba89-a37cdea1dd38","time":"2025-12-23T22:00:05.300933109Z"}]}
7662025-12-23T22:00:05.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7672025-12-23T22:00:05.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7682025-12-23T22:00:05.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7692025-12-23T22:00:05.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7702025-12-23T22:00:05.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7712025-12-23T22:00:05.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
7722025-12-23T22:00:05.364ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
7732025-12-23T22:00:05.365ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "9339e018-ba2a-4a5f-baa9-859db62c3fbc", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} }) sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
7742025-12-23T22:00:05.368ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = FirewallRules step id = Ensure
7752025-12-23T22:00:05.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7762025-12-23T22:00:05.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7772025-12-23T22:00:05.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7782025-12-23T22:00:05.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7792025-12-23T22:00:05.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7802025-12-23T22:00:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7812025-12-23T22:00:05.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7822025-12-23T22:00:05.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7832025-12-23T22:00:05.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7842025-12-23T22:00:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7852025-12-23T22:00:05.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7862025-12-23T22:00:05.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7872025-12-23T22:00:05.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7882025-12-23T22:00:05.388ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7892025-12-23T22:00:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7902025-12-23T22:00:05.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7912025-12-23T22:00:05.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7922025-12-23T22:00:05.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7932025-12-23T22:00:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7942025-12-23T22:00:05.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7952025-12-23T22:00:05.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7962025-12-23T22:00:05.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7972025-12-23T22:00:05.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7982025-12-23T22:00:05.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7992025-12-23T22:00:05.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8002025-12-23T22:00:05.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8012025-12-23T22:00:05.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8022025-12-23T22:00:05.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8032025-12-23T22:00:05.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8042025-12-23T22:00:05.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8052025-12-23T22:00:05.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8062025-12-23T22:00:05.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8072025-12-23T22:00:05.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8082025-12-23T22:00:05.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8092025-12-23T22:00:05.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8102025-12-23T22:00:05.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8112025-12-23T22:00:05.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8122025-12-23T22:00:05.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8132025-12-23T22:00:05.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8142025-12-23T22:00:05.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8152025-12-23T22:00:05.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8162025-12-23T22:00:05.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8172025-12-23T22:00:05.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8182025-12-23T22:00:05.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8192025-12-23T22:00:05.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8202025-12-23T22:00:05.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8212025-12-23T22:00:05.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8222025-12-23T22:00:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
8232025-12-23T22:00:05.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8242025-12-23T22:00:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8252025-12-23T22:00:05.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8262025-12-23T22:00:05.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8272025-12-23T22:00:05.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8282025-12-23T22:00:05.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8292025-12-23T22:00:05.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8302025-12-23T22:00:05.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8312025-12-23T22:00:05.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8322025-12-23T22:00:05.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8332025-12-23T22:00:05.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8342025-12-23T22:00:05.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8352025-12-23T22:00:05.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
8362025-12-23T22:00:05.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8372025-12-23T22:00:05.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8382025-12-23T22:00:05.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8392025-12-23T22:00:05.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8402025-12-23T22:00:05.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8412025-12-23T22:00:05.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8422025-12-23T22:00:05.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8432025-12-23T22:00:05.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8442025-12-23T22:00:05.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8452025-12-23T22:00:05.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8462025-12-23T22:00:05.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
8472025-12-23T22:00:05.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8482025-12-23T22:00:05.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8492025-12-23T22:00:05.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8502025-12-23T22:00:05.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8512025-12-23T22:00:05.457ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8522025-12-23T22:00:05.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8532025-12-23T22:00:05.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8542025-12-23T22:00:05.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8552025-12-23T22:00:05.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8562025-12-23T22:00:05.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8572025-12-23T22:00:05.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8582025-12-23T22:00:05.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8592025-12-23T22:00:05.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8602025-12-23T22:00:05.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
8612025-12-23T22:00:05.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8622025-12-23T22:00:05.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8632025-12-23T22:00:05.468ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8642025-12-23T22:00:05.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8652025-12-23T22:00:05.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8662025-12-23T22:00:05.473ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
8672025-12-23T22:00:05.473ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8682025-12-23T22:00:05.473ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "99167ade-0aa3-4fcd-9af1-033b74663f72", "content-length": "372", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
8692025-12-23T22:00:05.473ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8702025-12-23T22:00:05.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8712025-12-23T22:00:05.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8722025-12-23T22:00:05.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8732025-12-23T22:00:05.479ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1503, tv_nsec: 579671468 } })
8742025-12-23T22:00:05.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8752025-12-23T22:00:05.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8762025-12-23T22:00:05.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8772025-12-23T22:00:05.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8782025-12-23T22:00:05.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8792025-12-23T22:00:05.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8802025-12-23T22:00:05.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8812025-12-23T22:00:05.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8822025-12-23T22:00:05.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8832025-12-23T22:00:05.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8842025-12-23T22:00:05.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8852025-12-23T22:00:05.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8862025-12-23T22:00:05.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8872025-12-23T22:00:05.507ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8882025-12-23T22:00:05.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8892025-12-23T22:00:05.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8902025-12-23T22:00:05.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8912025-12-23T22:00:05.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8922025-12-23T22:00:05.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8932025-12-23T22:00:05.510ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8942025-12-23T22:00:05.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8952025-12-23T22:00:05.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8962025-12-23T22:00:05.512ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8972025-12-23T22:00:05.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8982025-12-23T22:00:05.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8992025-12-23T22:00:05.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9002025-12-23T22:00:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
9012025-12-23T22:00:05.517ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
9022025-12-23T22:00:05.518ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "6c403d33-800c-47e7-87c0-a2c8e3fa8d46", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
9032025-12-23T22:00:05.520ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:506
9042025-12-23T22:00:05.520ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 879.233409ms
9052025-12-23T22:00:05.520ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 879.24626ms iteration = 3 status = {}
9062025-12-23T22:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
9072025-12-23T22:00:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9082025-12-23T22:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9092025-12-23T22:00:05.524ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
9102025-12-23T22:00:05.524ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
9112025-12-23T22:00:05.524ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = e5cd2717-9137-4d76-a7a3-9e62c9d081c8 parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
9122025-12-23T22:00:05.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
9132025-12-23T22:00:05.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
9142025-12-23T22:00:05.526ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
9152025-12-23T22:00:05.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9162025-12-23T22:00:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4f63ad96-aac1-44f9-b14c-c672e54ceb6c", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9172025-12-23T22:00:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9182025-12-23T22:00:05.529ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "418e5cdd-b460-4134-a172-941c414d7c2d", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9192025-12-23T22:00:05.529ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9202025-12-23T22:00:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
9212025-12-23T22:00:05.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
9222025-12-23T22:00:05.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
9232025-12-23T22:00:05.533ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
9242025-12-23T22:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9252025-12-23T22:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9262025-12-23T22:00:05.534ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "67f46b00-19c7-4389-b2b1-e868c2140218", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9272025-12-23T22:00:05.534ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9282025-12-23T22:00:05.535ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
9292025-12-23T22:00:05.535ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
9302025-12-23T22:00:05.535ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 894.925262ms iteration = 25 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
9312025-12-23T22:00:05.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322025-12-23T22:00:05.536ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
9332025-12-23T22:00:05.536ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 917.047696ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
9342025-12-23T22:00:05.538ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
9352025-12-23T22:00:05.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9362025-12-23T22:00:05.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9372025-12-23T22:00:05.540ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
9382025-12-23T22:00:05.540ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
9392025-12-23T22:00:05.540ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
9402025-12-23T22:00:05.553ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
9412025-12-23T22:00:05.553ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
9422025-12-23T22:00:05.554ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ffd1fdca-07fe-43f3-9d3d-3f4152a6b4d0", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
9432025-12-23T22:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9442025-12-23T22:00:05.554ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1503, tv_nsec: 654097039 } })
9452025-12-23T22:00:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9462025-12-23T22:00:05.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9472025-12-23T22:00:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9482025-12-23T22:00:05.560ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), time_created: 2025-12-23T21:49:57.392343Z, time_modified: 2025-12-23T21:49:57.392343Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
9492025-12-23T22:00:05.560ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
9502025-12-23T22:00:05.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9512025-12-23T22:00:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9522025-12-23T22:00:05.564ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
9532025-12-23T22:00:05.565ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9542025-12-23T22:00:05.568ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a336e8c4-6f4b-4289-a9e3-a88a46368de5", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
9552025-12-23T22:00:05.571ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 289.029449ms iteration = 32 status = {}
9562025-12-23T22:00:05.633ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9572025-12-23T22:00:05.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9582025-12-23T22:00:05.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9592025-12-23T22:00:05.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9602025-12-23T22:00:05.636ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
9612025-12-23T22:00:05.636ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9622025-12-23T22:00:05.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9632025-12-23T22:00:05.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9642025-12-23T22:00:05.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9652025-12-23T22:00:05.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9662025-12-23T22:00:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
9672025-12-23T22:00:05.645ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
9682025-12-23T22:00:05.646ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "09b93638-704e-4359-8d85-5cc44377e393", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
9692025-12-23T22:00:05.647ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
9702025-12-23T22:00:05.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = Dns step id = Ensure
9712025-12-23T22:00:05.647ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
9722025-12-23T22:00:05.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9732025-12-23T22:00:05.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9742025-12-23T22:00:05.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9752025-12-23T22:00:05.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9762025-12-23T22:00:05.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9772025-12-23T22:00:05.650ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
9782025-12-23T22:00:05.650ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
9792025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
9802025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
9812025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
9822025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
9832025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
9842025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
9852025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
9862025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
9872025-12-23T22:00:05.651ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
9882025-12-23T22:00:05.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9892025-12-23T22:00:05.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9902025-12-23T22:00:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
9912025-12-23T22:00:05.654ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
9922025-12-23T22:00:05.654ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9932025-12-23T22:00:05.655ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
9942025-12-23T22:00:05.655ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
9952025-12-23T22:00:05.655ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
9962025-12-23T22:00:05.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9972025-12-23T22:00:05.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9982025-12-23T22:00:05.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9992025-12-23T22:00:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
10002025-12-23T22:00:05.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10012025-12-23T22:00:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10022025-12-23T22:00:05.667ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
10032025-12-23T22:00:05.667ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10042025-12-23T22:00:05.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10052025-12-23T22:00:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10062025-12-23T22:00:05.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10072025-12-23T22:00:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
10082025-12-23T22:00:05.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10092025-12-23T22:00:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10102025-12-23T22:00:05.675ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
10112025-12-23T22:00:05.675ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
10122025-12-23T22:00:05.675ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
10132025-12-23T22:00:05.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10142025-12-23T22:00:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10152025-12-23T22:00:05.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10162025-12-23T22:00:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10172025-12-23T22:00:05.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10182025-12-23T22:00:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
10192025-12-23T22:00:05.683ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
10202025-12-23T22:00:05.683ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
10212025-12-23T22:00:05.683ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
10222025-12-23T22:00:05.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10232025-12-23T22:00:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10242025-12-23T22:00:05.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10252025-12-23T22:00:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
10262025-12-23T22:00:05.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10272025-12-23T22:00:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10282025-12-23T22:00:05.691ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
10292025-12-23T22:00:05.691ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
10302025-12-23T22:00:05.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10312025-12-23T22:00:05.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10322025-12-23T22:00:05.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332025-12-23T22:00:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
10342025-12-23T22:00:05.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10352025-12-23T22:00:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10362025-12-23T22:00:05.698ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
10372025-12-23T22:00:05.698ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
10382025-12-23T22:00:05.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10392025-12-23T22:00:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10402025-12-23T22:00:05.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10412025-12-23T22:00:05.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
10422025-12-23T22:00:05.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10432025-12-23T22:00:05.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10442025-12-23T22:00:05.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10452025-12-23T22:00:05.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10462025-12-23T22:00:05.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10472025-12-23T22:00:05.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10482025-12-23T22:00:05.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10492025-12-23T22:00:05.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
10502025-12-23T22:00:05.714ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
10512025-12-23T22:00:05.714ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
10522025-12-23T22:00:05.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = OmicronZones step id = Cleanup
10532025-12-23T22:00:05.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = OmicronZones step id = Cleanup
10542025-12-23T22:00:05.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = PhysicalDisks step id = Cleanup
10552025-12-23T22:00:05.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = Clickhouse step id = Ensure
10562025-12-23T22:00:05.715ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = Clickhouse step id = Ensure
10572025-12-23T22:00:05.756ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
10582025-12-23T22:00:05.761ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "764bfd34-0b48-4159-9c4c-cd3a3d2bd2c2", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
10592025-12-23T22:00:05.762ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = SupportBundles step id = Cleanup
10602025-12-23T22:00:05.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10612025-12-23T22:00:05.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10622025-12-23T22:00:05.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10632025-12-23T22:00:05.766ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
10642025-12-23T22:00:05.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10652025-12-23T22:00:05.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10662025-12-23T22:00:05.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10672025-12-23T22:00:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10682025-12-23T22:00:05.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10692025-12-23T22:00:05.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
10702025-12-23T22:00:05.812ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = OmicronZones step id = Cleanup
10712025-12-23T22:00:05.812ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
10722025-12-23T22:00:05.812ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
10732025-12-23T22:00:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10742025-12-23T22:00:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10752025-12-23T22:00:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10762025-12-23T22:00:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
10772025-12-23T22:00:05.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10782025-12-23T22:00:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10792025-12-23T22:00:05.817ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
10802025-12-23T22:00:05.817ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(02039deb-aee8-405c-80d5-e584bd199fe1 (blueprint), false)
10812025-12-23T22:00:05.818ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = Cockroach step id = Ensure
10822025-12-23T22:00:05.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
10832025-12-23T22:00:05.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
10842025-12-23T22:00:05.833ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
10852025-12-23T22:00:05.833ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 7cfcab5b-11ab-4ae0-8d50-d6827e2995a7 step component = MgsUpdates step id = Ensure
10862025-12-23T22:00:05.833ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 32 reason = Dependency
10872025-12-23T22:00:05.833ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Dependency
10882025-12-23T22:00:05.833ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10892025-12-23T22:00:05.833ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10902025-12-23T22:00:05.834ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.214296137s iteration = 12 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3240,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":141061622,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":141061572,"secs":0}},"event_index":1,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":141073442,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":69506042,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":69505982,"secs":0}},"event_index":2,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":210597525,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":162060735,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":162060695,"secs":0}},"event_index":3,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":372672800,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":375598193,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":375598153,"secs":0}},"event_index":4,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":748294783,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":279519100,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":279519060,"secs":0}},"event_index":5,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":27825283,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":67120745,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":67120705,"secs":0}},"event_index":6,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":94952178,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4600,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4560,"secs":0}},"event_index":7,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":94970418,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2751,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2700,"secs":0}},"event_index":8,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":94976729,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3840,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3790,"secs":0}},"event_index":9,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":94982389,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2130,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2080,"secs":0}},"event_index":10,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":94990529,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47327256,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":47327206,"secs":0}},"event_index":11,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":142318755,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50220548,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50220508,"secs":0}},"event_index":12,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":192550293,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5401883,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5401843,"secs":0}},"event_index":13,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":197968326,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15841387,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15841347,"secs":0}},"event_index":14,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":213816253,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5070,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5030,"secs":0}},"event_index":15,"execution_id":"a2e04157-125b-4677-a054-b2ef2fd29476","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":213829773,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"02039deb-aee8-405c-80d5-e584bd199fe1"}
10912025-12-23T22:00:05.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10922025-12-23T22:00:05.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10932025-12-23T22:00:05.836ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10942025-12-23T22:00:05.836ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10952025-12-23T22:00:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10962025-12-23T22:00:05.839ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10972025-12-23T22:00:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10982025-12-23T22:00:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
10992025-12-23T22:00:05.839ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11002025-12-23T22:00:05.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
11012025-12-23T22:00:05.843ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(737a4a86-a2e2-49a1-897a-b4bcbf7fa821, 0), (fe383fa9-2503-4269-a0d4-94d26b68edd3, 0), (f948b43c-60d4-4f42-abfd-49b154566105, 0)]
11022025-12-23T22:00:05.843ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1503, tv_nsec: 943593880 } })
11032025-12-23T22:00:05.877ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("01b204c0-22af-4353-ad94-4c2c0d1eb01b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70f89219-475c-48c6-a8c8-388723cccdcf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("7b633ce6-0ad9-42aa-8164-bc5c5f301b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("eaafa87a-91ad-4109-aa73-7065c531ed97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1503, tv_nsec: 976739306 } })
11042025-12-23T22:00:05.877ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
11052025-12-23T22:00:05.877ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
11062025-12-23T22:00:05.877ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
11072025-12-23T22:00:05.878ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9288b753-3f91-409c-90ab-c3e799fdb55d", "content-length": "476", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11082025-12-23T22:00:05.878ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
11092025-12-23T22:00:05.878ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0ba74fc-0faa-4db1-8cd4-bf21344a3c89", "content-length": "734", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11102025-12-23T22:00:05.878ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
11112025-12-23T22:00:05.879ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa30cda9-241a-4d24-9e73-d45487b16f70", "content-length": "111", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11122025-12-23T22:00:05.879ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
11132025-12-23T22:00:05.879ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f3de9bd-553b-4469-ac7b-4fbd247641f8", "content-length": "111", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11142025-12-23T22:00:05.879ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
11152025-12-23T22:00:05.879ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f18c276-c26b-4913-aba3-755593f94647", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11162025-12-23T22:00:05.880ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
11172025-12-23T22:00:05.880ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc33edb-a4dc-41c6-8dee-76def239965e", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11182025-12-23T22:00:05.880ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
11192025-12-23T22:00:05.880ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91cfaf9a-b99f-49c2-9865-7d22304b6515", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11202025-12-23T22:00:05.880ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
11212025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5319438c-9689-4dc6-a3cf-7418415178f3", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11222025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
11232025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c50f82dc-4e8d-4b35-982a-a04e090e4583", "content-length": "702", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11242025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
11252025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dfa6a5b-4c11-49d6-95d1-ecd69f10f4a6", "content-length": "727", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11262025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
11272025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2335182-9cbc-4230-8fd8-58dcc43a2479", "content-length": "729", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11282025-12-23T22:00:05.881ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
11292025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8ed456c-265c-44cd-8d21-0af2b891c486", "content-length": "728", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11302025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
11312025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7af80b48-3e16-4bc1-a73e-682668eb5cc4", "content-length": "722", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11322025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
11332025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "8300f6a1-3359-440d-8777-8372113c96f9", "content-length": "10", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11342025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
11352025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60e81b01-abe0-4631-bd69-d0de0c6dcb32", "content-length": "144", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11362025-12-23T22:00:05.882ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
11372025-12-23T22:00:05.883ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4da8bf3-b9ed-42b1-ae37-b12358a4acc4", "content-length": "147", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11382025-12-23T22:00:05.883ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
11392025-12-23T22:00:05.883ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0146d96-bc1a-44ba-a8a9-df38c142397e", "content-length": "109", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11402025-12-23T22:00:05.883ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
11412025-12-23T22:00:05.883ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "19dbda46-54dd-4be8-b9e7-ad722f0e59f5", "content-length": "109", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11422025-12-23T22:00:05.883ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
11432025-12-23T22:00:05.884ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54948164-ae8a-4e23-ad25-4a35f852e909", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11442025-12-23T22:00:05.884ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
11452025-12-23T22:00:05.884ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f25d0e99-1684-48eb-a838-9e7a5fb0ecf8", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11462025-12-23T22:00:05.884ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
11472025-12-23T22:00:05.884ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38a3e4e3-457d-4696-9b45-7076507fc979", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11482025-12-23T22:00:05.884ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
11492025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08828187-4394-400a-90a8-c701d29b9c5e", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11502025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
11512025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fb88f83-9e7e-4c05-b663-62e3c1676fa5", "content-length": "702", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11522025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
11532025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b597b70e-e59b-44d2-b793-b4c1775a68b5", "content-length": "727", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11542025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
11552025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ae39c07-c2ac-46de-80ad-31d1bcd5ad95", "content-length": "729", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11562025-12-23T22:00:05.885ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
11572025-12-23T22:00:05.886ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52dba17b-f169-41b9-a39f-ee1a402dc7a4", "content-length": "728", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11582025-12-23T22:00:05.886ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11592025-12-23T22:00:05.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
11602025-12-23T22:00:05.888ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11612025-12-23T22:00:05.891ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
11622025-12-23T22:00:05.891ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11632025-12-23T22:00:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
11642025-12-23T22:00:05.893ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11652025-12-23T22:00:05.893ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11662025-12-23T22:00:05.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11672025-12-23T22:00:05.895ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11682025-12-23T22:00:05.895ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
11692025-12-23T22:00:05.895ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11702025-12-23T22:00:05.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
11712025-12-23T22:00:05.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11722025-12-23T22:00:05.897ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11732025-12-23T22:00:05.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11742025-12-23T22:00:05.930ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
11752025-12-23T22:00:05.930ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
11762025-12-23T22:00:05.947ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
11772025-12-23T22:00:05.947ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 113.164685ms iteration = 32 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11782025-12-23T22:00:05.955ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "583339a2-1e56-4224-9741-2b6886d0e2b1", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11792025-12-23T22:00:05.955ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
11802025-12-23T22:00:05.955ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
11812025-12-23T22:00:05.955ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea3debe0-59ef-4196-a7df-4f4fcf831f0c", "content-length": "682", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11822025-12-23T22:00:05.955ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11832025-12-23T22:00:05.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11842025-12-23T22:00:05.959ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11852025-12-23T22:00:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11862025-12-23T22:00:05.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11872025-12-23T22:00:05.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11882025-12-23T22:00:05.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11892025-12-23T22:00:05.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11902025-12-23T22:00:05.973ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.216488628s iteration = 21 status = {}
11912025-12-23T22:00:05.986ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "de809614-6711-4c16-b04d-9a4e195d8ad0", "content-length": "95743", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
11922025-12-23T22:00:05.988ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
11932025-12-23T22:00:05.988ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
11942025-12-23T22:00:05.988ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
11952025-12-23T22:00:05.988ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
11962025-12-23T22:00:05.988ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
11972025-12-23T22:00:05.988ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
11982025-12-23T22:00:05.988ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
11992025-12-23T22:00:05.988ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
12002025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddc92b39-e978-4977-8695-c3942dbd2ec3", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
12012025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
12022025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "329fa492-4b05-4cd4-8683-5c43e0fe2e23", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
12032025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5edac551-4fd8-45db-af36-1043941cc46b", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
12042025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e86ddbc-5b04-45e3-a5b0-f2f5968a9340", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
12052025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "39c93bca-149a-4a90-a0d3-ffea93fb3e45", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:05 GMT"} })
12062025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
12072025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
12082025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
12092025-12-23T22:00:05.989ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
12102025-12-23T22:00:06.031ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e556ff-8355-4325-86d8-b4f70b463ad4", "content-length": "412389", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12112025-12-23T22:00:06.032ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "702fb3f4-b1aa-4ff9-8c04-f365addc4f34", "content-length": "427475", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12122025-12-23T22:00:06.032ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ac435d5-e5c3-41ac-85b5-68a769abf662", "content-length": "414342", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12132025-12-23T22:00:06.033ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0776e63-f7d4-43a2-82ae-c99c08d239e1", "content-length": "407314", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12142025-12-23T22:00:06.033ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bad4498-c99c-4f76-9db5-cac8709a62a3", "content-length": "407156", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12152025-12-23T22:00:06.036ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
12162025-12-23T22:00:06.036ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
12172025-12-23T22:00:06.036ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
12182025-12-23T22:00:06.036ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
12192025-12-23T22:00:06.036ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
12202025-12-23T22:00:06.072ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c (omicron_zone)
12212025-12-23T22:00:06.072ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
12222025-12-23T22:00:06.075ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "824b1ff8-781c-4824-8aef-ab80146ab9db", "content-length": "128", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12232025-12-23T22:00:06.078ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
12242025-12-23T22:00:06.179ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
12252025-12-23T22:00:06.179ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
12262025-12-23T22:00:06.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "abcd1ae4-2ff6-45d6-bb63-ecba08d598e1", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12272025-12-23T22:00:06.180ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
12282025-12-23T22:00:06.182ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
12292025-12-23T22:00:06.182ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
12302025-12-23T22:00:06.183ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "278e5fe4-00a2-49e5-9197-5f3bb1459585", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12312025-12-23T22:00:06.183ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
12322025-12-23T22:00:06.185ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
12332025-12-23T22:00:06.185ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
12342025-12-23T22:00:06.185ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a116b453-d786-43b4-8b8a-bac4bdfc5ac8", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:00:06 GMT"} })
12352025-12-23T22:00:06.186ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
12362025-12-23T22:00:06.187ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
12372025-12-23T22:00:06.187ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
12382025-12-23T22:00:06.189ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12392025-12-23T22:00:06.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
12402025-12-23T22:00:06.191ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12412025-12-23T22:00:06.194ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
12422025-12-23T22:00:06.194ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
12432025-12-23T22:00:06.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
12442025-12-23T22:00:06.305ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
12452025-12-23T22:00:06.305ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7 time_started = 2025-12-23 22:00:05.877727 UTC
12462025-12-23T22:00:06.305ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 54 reason = Dependency
12472025-12-23T22:00:06.305ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 36 reason = Dependency
12482025-12-23T22:00:06.305ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
12492025-12-23T22:00:06.305ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
12502025-12-23T22:00:06.305ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 24.981\u{b5}s iteration = 36 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
12512025-12-23T22:00:06.305ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 471.919772ms iteration = 13 status = {"collection_id":"51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7","time_done":"2025-12-23 22:00:06.187666 UTC","time_started":"2025-12-23 22:00:05.877727 UTC"}
12522025-12-23T22:00:06.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
12532025-12-23T22:00:06.308ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12542025-12-23T22:00:06.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
12552025-12-23T22:00:06.311ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
12562025-12-23T22:00:06.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
12572025-12-23T22:00:06.314ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
12582025-12-23T22:00:06.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
12592025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 37 reason = Dependency
12602025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 59.609383ms iteration = 54 status = {"Loaded":{"collection_id":"51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7","time_loaded":"2025-12-23T22:00:06.305819635Z","time_started":"2025-12-23T22:00:05.877727Z"}}
12612025-12-23T22:00:06.365ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
12622025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 16 reason = Dependency
12632025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 26 reason = Dependency
12642025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Dependency
12652025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 28.921\u{b5}s iteration = 37 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
12662025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12672025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12682025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12692025-12-23T22:00:06.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12702025-12-23T22:00:06.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12712025-12-23T22:00:06.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12722025-12-23T22:00:06.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12732025-12-23T22:00:06.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12742025-12-23T22:00:06.370ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12752025-12-23T22:00:06.372ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
12762025-12-23T22:00:06.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12772025-12-23T22:00:06.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12782025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12792025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
12802025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12812025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12822025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12832025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12842025-12-23T22:00:06.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12852025-12-23T22:00:06.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12862025-12-23T22:00:06.376ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12872025-12-23T22:00:06.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12882025-12-23T22:00:06.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
12892025-12-23T22:00:06.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12902025-12-23T22:00:06.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12912025-12-23T22:00:06.378ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
12922025-12-23T22:00:06.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12932025-12-23T22:00:06.378ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12942025-12-23T22:00:06.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12952025-12-23T22:00:06.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
12962025-12-23T22:00:06.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
12972025-12-23T22:00:06.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12982025-12-23T22:00:06.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12992025-12-23T22:00:06.381ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13002025-12-23T22:00:06.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13012025-12-23T22:00:06.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13022025-12-23T22:00:06.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13032025-12-23T22:00:06.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13042025-12-23T22:00:06.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13052025-12-23T22:00:06.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13062025-12-23T22:00:06.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13072025-12-23T22:00:06.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13082025-12-23T22:00:06.386ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
13092025-12-23T22:00:06.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13102025-12-23T22:00:06.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13112025-12-23T22:00:06.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13122025-12-23T22:00:06.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13132025-12-23T22:00:06.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13142025-12-23T22:00:06.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13152025-12-23T22:00:06.389ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
13162025-12-23T22:00:06.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13172025-12-23T22:00:06.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13182025-12-23T22:00:06.390ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
13192025-12-23T22:00:06.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13202025-12-23T22:00:06.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13212025-12-23T22:00:06.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13222025-12-23T22:00:06.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13232025-12-23T22:00:06.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13242025-12-23T22:00:06.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13252025-12-23T22:00:06.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13262025-12-23T22:00:06.399ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
13272025-12-23T22:00:06.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
13282025-12-23T22:00:06.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
13292025-12-23T22:00:06.402ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
13302025-12-23T22:00:06.402ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 36.649315ms iteration = 16 status = {"blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","inventory_collection_id":"51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
13312025-12-23T22:00:06.434ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 68.625797ms iteration = 34 status = {"physical_disks_added":0}
13322025-12-23T22:00:06.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13332025-12-23T22:00:06.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13342025-12-23T22:00:06.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13352025-12-23T22:00:06.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13362025-12-23T22:00:06.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13372025-12-23T22:00:06.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
13382025-12-23T22:00:06.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13392025-12-23T22:00:06.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13402025-12-23T22:00:06.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13412025-12-23T22:00:06.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13422025-12-23T22:00:06.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13432025-12-23T22:00:06.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
13442025-12-23T22:00:06.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13452025-12-23T22:00:06.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13462025-12-23T22:00:06.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13472025-12-23T22:00:06.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
13482025-12-23T22:00:06.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13492025-12-23T22:00:06.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13502025-12-23T22:00:06.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13512025-12-23T22:00:06.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13522025-12-23T22:00:06.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13532025-12-23T22:00:06.468ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
13542025-12-23T22:00:06.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13552025-12-23T22:00:06.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13562025-12-23T22:00:06.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13572025-12-23T22:00:06.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13582025-12-23T22:00:06.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13592025-12-23T22:00:06.475ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
13602025-12-23T22:00:06.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13612025-12-23T22:00:06.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13622025-12-23T22:00:06.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13632025-12-23T22:00:06.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13642025-12-23T22:00:06.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13652025-12-23T22:00:06.481ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
13662025-12-23T22:00:06.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13672025-12-23T22:00:06.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13682025-12-23T22:00:06.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13692025-12-23T22:00:06.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13702025-12-23T22:00:06.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13712025-12-23T22:00:06.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13722025-12-23T22:00:06.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13732025-12-23T22:00:06.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
13742025-12-23T22:00:06.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13752025-12-23T22:00:06.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13762025-12-23T22:00:06.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13772025-12-23T22:00:06.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
13782025-12-23T22:00:06.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13792025-12-23T22:00:06.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13802025-12-23T22:00:06.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13812025-12-23T22:00:06.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13822025-12-23T22:00:06.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13832025-12-23T22:00:06.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13842025-12-23T22:00:06.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13852025-12-23T22:00:06.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
13862025-12-23T22:00:06.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13872025-12-23T22:00:06.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13882025-12-23T22:00:06.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13892025-12-23T22:00:06.524ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
13902025-12-23T22:00:06.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13912025-12-23T22:00:06.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13922025-12-23T22:00:06.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13932025-12-23T22:00:06.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13942025-12-23T22:00:06.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13952025-12-23T22:00:06.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13962025-12-23T22:00:06.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13972025-12-23T22:00:06.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
13982025-12-23T22:00:06.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13992025-12-23T22:00:06.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14002025-12-23T22:00:06.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14012025-12-23T22:00:06.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14022025-12-23T22:00:06.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14032025-12-23T22:00:06.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
14042025-12-23T22:00:06.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14052025-12-23T22:00:06.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14062025-12-23T22:00:06.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14072025-12-23T22:00:06.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14082025-12-23T22:00:06.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14092025-12-23T22:00:06.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
14102025-12-23T22:00:06.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14112025-12-23T22:00:06.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14122025-12-23T22:00:06.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14132025-12-23T22:00:06.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14142025-12-23T22:00:06.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14152025-12-23T22:00:06.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
14162025-12-23T22:00:06.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14172025-12-23T22:00:06.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14182025-12-23T22:00:06.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14192025-12-23T22:00:06.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14202025-12-23T22:00:06.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14212025-12-23T22:00:06.563ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
14222025-12-23T22:00:06.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14232025-12-23T22:00:06.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14242025-12-23T22:00:06.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14252025-12-23T22:00:06.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14262025-12-23T22:00:06.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272025-12-23T22:00:06.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
14282025-12-23T22:00:06.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14292025-12-23T22:00:06.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14302025-12-23T22:00:06.571ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
14312025-12-23T22:00:06.571ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
14322025-12-23T22:00:06.571ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9440eff9-e529-42e3-a4c5-084f6a1be96e parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
14332025-12-23T22:00:06.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14342025-12-23T22:00:06.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14352025-12-23T22:00:06.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14362025-12-23T22:00:06.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
14372025-12-23T22:00:06.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14382025-12-23T22:00:06.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14392025-12-23T22:00:06.579ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
14402025-12-23T22:00:06.579ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
14412025-12-23T22:00:06.579ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 213.680041ms iteration = 26 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
14422025-12-23T22:00:08.923ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Timeout
14432025-12-23T22:00:08.924ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14442025-12-23T22:00:08.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
14452025-12-23T22:00:08.926ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14462025-12-23T22:00:08.929ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
14472025-12-23T22:00:08.929ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14482025-12-23T22:00:08.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
14492025-12-23T22:00:08.931ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(a35b06f2-3902-4259-8473-ff842f4362c9, 0), (1e766c62-a2ca-449d-8c26-c75361baac79, 0), (51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7, 0), (c7d335e4-0197-415a-bfdb-e9e304196cc2, 0)] collection_id = a35b06f2-3902-4259-8473-ff842f4362c9
14502025-12-23T22:00:08.931ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14512025-12-23T22:00:08.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
14522025-12-23T22:00:09.037ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = a35b06f2-3902-4259-8473-ff842f4362c9 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
14532025-12-23T22:00:09.037ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14542025-12-23T22:00:09.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
14552025-12-23T22:00:09.041ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(1e766c62-a2ca-449d-8c26-c75361baac79, 0), (51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7, 0), (c7d335e4-0197-415a-bfdb-e9e304196cc2, 0), (44ce8ae2-e1ec-40b1-b571-85365dc3d6dd, 0)] collection_id = 1e766c62-a2ca-449d-8c26-c75361baac79
14562025-12-23T22:00:09.041ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14572025-12-23T22:00:09.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
14582025-12-23T22:00:09.158ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 1e766c62-a2ca-449d-8c26-c75361baac79 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
14592025-12-23T22:00:09.158ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
14602025-12-23T22:00:09.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
14612025-12-23T22:00:09.163ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(51e8dd18-b0d8-42eb-a8b1-da84c0dbb6f7, 0), (c7d335e4-0197-415a-bfdb-e9e304196cc2, 0), (44ce8ae2-e1ec-40b1-b571-85365dc3d6dd, 0)]
14622025-12-23T22:00:09.163ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1507, tv_nsec: 263587817 } })
14632025-12-23T22:00:09.197ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("01b204c0-22af-4353-ad94-4c2c0d1eb01b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70f89219-475c-48c6-a8c8-388723cccdcf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("7b633ce6-0ad9-42aa-8164-bc5c5f301b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("eaafa87a-91ad-4109-aa73-7065c531ed97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1507, tv_nsec: 297077282 } })
14642025-12-23T22:00:09.197ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
14652025-12-23T22:00:09.197ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
14662025-12-23T22:00:09.197ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
14672025-12-23T22:00:09.198ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e011e1de-dbf0-41ce-980b-1828ecd6470b", "content-length": "476", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14682025-12-23T22:00:09.198ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
14692025-12-23T22:00:09.198ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "88215dfb-afeb-4e8a-82f1-7273a3d7d7ab", "content-length": "734", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14702025-12-23T22:00:09.198ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
14712025-12-23T22:00:09.199ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af3d3cc1-fda8-4799-8825-a59c718e1f65", "content-length": "111", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14722025-12-23T22:00:09.199ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
14732025-12-23T22:00:09.199ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d87e08af-fab2-4b2f-aa6f-eaeeb65da3c8", "content-length": "111", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14742025-12-23T22:00:09.199ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
14752025-12-23T22:00:09.199ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "208af369-24b3-484a-80a5-90cd7174f0fc", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14762025-12-23T22:00:09.199ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
14772025-12-23T22:00:09.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f3b899a-8279-4f1c-b4a8-8a16bfa1ca8a", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14782025-12-23T22:00:09.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
14792025-12-23T22:00:09.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97fa8274-6843-4359-93b1-e019e40afca2", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14802025-12-23T22:00:09.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
14812025-12-23T22:00:09.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "417270a4-f185-4020-821c-eb27c553f77d", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14822025-12-23T22:00:09.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
14832025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5645a0b5-cc44-4e71-aa1c-3c9b393ee0bb", "content-length": "702", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14842025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14852025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a57020d-42da-412e-9a6c-c1741aa65f43", "content-length": "727", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14862025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14872025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3f4a117-e7d3-4c3d-8b40-7f77a66137d8", "content-length": "729", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14882025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
14892025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52074d2f-0ced-4c9f-a757-0575b8604098", "content-length": "728", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14902025-12-23T22:00:09.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
14912025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6072bc7-1569-458f-932f-98c06e4184be", "content-length": "722", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14922025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
14932025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbc70570-be11-4e44-8cb7-4835ddda4229", "content-length": "10", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14942025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
14952025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60bc5fe8-3421-43b7-bcb0-d42ce246a828", "content-length": "144", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14962025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
14972025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d8b688c-a252-4d9e-980f-95dab77bb42e", "content-length": "147", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
14982025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
14992025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a7610e1-5bdd-43f7-9644-b35c71c5be6c", "content-length": "109", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15002025-12-23T22:00:09.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
15012025-12-23T22:00:09.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8e95346-dd77-4a61-b92d-acb412e4e353", "content-length": "109", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15022025-12-23T22:00:09.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
15032025-12-23T22:00:09.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d562163-c5a8-4b24-87c8-193141dadeab", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15042025-12-23T22:00:09.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
15052025-12-23T22:00:09.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "368ad1c5-45ee-4053-9a4d-c5513727ec07", "content-length": "163", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15062025-12-23T22:00:09.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
15072025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cacee0bd-cfe4-4f87-911c-c9300d682e90", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15082025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
15092025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "49c95908-73fd-4088-836f-9f83b31b9b71", "content-length": "166", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15102025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
15112025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9971e9ad-65ac-43be-bb9a-cabae0997e7d", "content-length": "702", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15122025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
15132025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bcac235-7157-482e-ba38-e199580ac3be", "content-length": "727", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15142025-12-23T22:00:09.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
15152025-12-23T22:00:09.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f1f47ed-c7a0-4766-8cd1-c6b4dbe89e01", "content-length": "729", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15162025-12-23T22:00:09.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
15172025-12-23T22:00:09.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad72a133-ce45-4a8a-9e9c-efb4e2c53e45", "content-length": "728", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15182025-12-23T22:00:09.205ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15192025-12-23T22:00:09.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15202025-12-23T22:00:09.207ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15212025-12-23T22:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
15222025-12-23T22:00:09.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15232025-12-23T22:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15242025-12-23T22:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15252025-12-23T22:00:09.213ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
15262025-12-23T22:00:09.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15272025-12-23T22:00:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15282025-12-23T22:00:09.246ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
15292025-12-23T22:00:09.246ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
15302025-12-23T22:00:09.295ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c614f0da-3029-4ddd-addf-5b217c04e603", "content-length": "95743", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15312025-12-23T22:00:09.296ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
15322025-12-23T22:00:09.296ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
15332025-12-23T22:00:09.296ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
15342025-12-23T22:00:09.296ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
15352025-12-23T22:00:09.296ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
15362025-12-23T22:00:09.296ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
15372025-12-23T22:00:09.296ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
15382025-12-23T22:00:09.296ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
15392025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "682a8901-de09-4ff1-99a9-657a2780ab7a", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15402025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc972445-d920-4a31-9de3-6242631b8869", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15412025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cdd5527-215c-4d26-a4ad-fdcf4faa0342", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15422025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
15432025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
15442025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
15452025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b82228f2-8f46-448c-98c9-ad33c06e4013", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15462025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
15472025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6934d7-4b0b-4ef9-965b-a5917501e22b", "content-length": "64", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15482025-12-23T22:00:09.297ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
15492025-12-23T22:00:09.334ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4589721-be1d-4e77-936f-0cf6ad936a91", "content-length": "412635", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15502025-12-23T22:00:09.334ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "941838b0-5510-4e92-8c14-9db42c7ae73f", "content-length": "407617", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15512025-12-23T22:00:09.335ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "309647a5-d7b0-4215-a7ac-ce9a2e4e4473", "content-length": "414987", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15522025-12-23T22:00:09.335ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2e3fa00-3ec5-4e60-a773-1fe842a8881f", "content-length": "407508", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15532025-12-23T22:00:09.335ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9661ac44-312b-4941-b685-c4d6cd84819e", "content-length": "427577", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15542025-12-23T22:00:09.338ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
15552025-12-23T22:00:09.338ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
15562025-12-23T22:00:09.338ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
15572025-12-23T22:00:09.338ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
15582025-12-23T22:00:09.338ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
15592025-12-23T22:00:09.368ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c (omicron_zone)
15602025-12-23T22:00:09.368ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
15612025-12-23T22:00:09.371ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aea8766-4ee8-473d-b9cb-2e5a48e879c3", "content-length": "128", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15622025-12-23T22:00:09.372ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
15632025-12-23T22:00:09.461ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
15642025-12-23T22:00:09.461ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
15652025-12-23T22:00:09.461ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b3edca1-bfed-453e-8443-cdd03d4f910d", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15662025-12-23T22:00:09.461ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
15672025-12-23T22:00:09.463ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
15682025-12-23T22:00:09.463ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
15692025-12-23T22:00:09.464ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ee2ba67-558a-43a9-8ce4-7a133907c045", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15702025-12-23T22:00:09.464ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
15712025-12-23T22:00:09.465ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
15722025-12-23T22:00:09.465ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
15732025-12-23T22:00:09.465ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "17961b4f-1c23-4677-8bac-87394bcc6210", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:00:09 GMT"} })
15742025-12-23T22:00:09.465ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
15752025-12-23T22:00:09.467ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
15762025-12-23T22:00:09.468ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
15772025-12-23T22:00:09.470ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15782025-12-23T22:00:09.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15792025-12-23T22:00:09.472ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15802025-12-23T22:00:09.474ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
15812025-12-23T22:00:09.474ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15822025-12-23T22:00:09.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15832025-12-23T22:00:09.678ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
15842025-12-23T22:00:09.678ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 time_started = 2025-12-23 22:00:09.197906 UTC
15852025-12-23T22:00:09.678ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 55 reason = Dependency
15862025-12-23T22:00:09.678ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 38 reason = Dependency
15872025-12-23T22:00:09.678ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
15882025-12-23T22:00:09.678ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
15892025-12-23T22:00:09.678ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 25.131\u{b5}s iteration = 38 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
15902025-12-23T22:00:09.678ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 754.919299ms iteration = 14 status = {"collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","time_done":"2025-12-23 22:00:09.467991 UTC","time_started":"2025-12-23 22:00:09.197906 UTC"}
15912025-12-23T22:00:09.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
15922025-12-23T22:00:09.681ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15932025-12-23T22:00:09.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
15942025-12-23T22:00:09.685ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
15952025-12-23T22:00:09.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
15962025-12-23T22:00:09.688ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
15972025-12-23T22:00:09.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
15982025-12-23T22:00:09.752ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 27 reason = Dependency
15992025-12-23T22:00:09.752ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 74.093271ms iteration = 55 status = {"Loaded":{"collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","time_loaded":"2025-12-23T22:00:09.678817877Z","time_started":"2025-12-23T22:00:09.197906Z"}}
16002025-12-23T22:00:09.752ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 17 reason = Dependency
16012025-12-23T22:00:09.752ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Dependency
16022025-12-23T22:00:09.752ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 39 reason = Dependency
16032025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16042025-12-23T22:00:09.753ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
16052025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16062025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.601\u{b5}s iteration = 39 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
16072025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16082025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16092025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16102025-12-23T22:00:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16112025-12-23T22:00:09.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16122025-12-23T22:00:09.755ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16132025-12-23T22:00:09.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16142025-12-23T22:00:09.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 122 reason = Timeout
16152025-12-23T22:00:09.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
16162025-12-23T22:00:09.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
16172025-12-23T22:00:09.759ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16182025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16192025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16202025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16212025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16222025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16232025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16242025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
16252025-12-23T22:00:09.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16262025-12-23T22:00:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16272025-12-23T22:00:09.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16282025-12-23T22:00:09.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16292025-12-23T22:00:09.763ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16302025-12-23T22:00:09.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16312025-12-23T22:00:09.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
16322025-12-23T22:00:09.765ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
16332025-12-23T22:00:09.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
16342025-12-23T22:00:09.765ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16352025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16362025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16372025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16382025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
16392025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16402025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16412025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
16422025-12-23T22:00:09.767ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.167163ms iteration = 122 status = {"config_updated":false}
16432025-12-23T22:00:09.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16442025-12-23T22:00:09.769ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16452025-12-23T22:00:09.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16462025-12-23T22:00:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16472025-12-23T22:00:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16482025-12-23T22:00:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16492025-12-23T22:00:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16502025-12-23T22:00:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16512025-12-23T22:00:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16522025-12-23T22:00:09.775ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16532025-12-23T22:00:09.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16542025-12-23T22:00:09.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16552025-12-23T22:00:09.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
16562025-12-23T22:00:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16572025-12-23T22:00:09.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16582025-12-23T22:00:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16592025-12-23T22:00:09.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16602025-12-23T22:00:09.780ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 27.368333ms iteration = 35 status = {"physical_disks_added":0}
16612025-12-23T22:00:09.781ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16622025-12-23T22:00:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16632025-12-23T22:00:09.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16642025-12-23T22:00:09.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
16652025-12-23T22:00:09.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16662025-12-23T22:00:09.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16672025-12-23T22:00:09.784ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
16682025-12-23T22:00:09.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16692025-12-23T22:00:09.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16702025-12-23T22:00:09.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16712025-12-23T22:00:09.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16722025-12-23T22:00:09.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16732025-12-23T22:00:09.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16742025-12-23T22:00:09.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16752025-12-23T22:00:09.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16762025-12-23T22:00:09.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16772025-12-23T22:00:09.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16782025-12-23T22:00:09.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16792025-12-23T22:00:09.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16802025-12-23T22:00:09.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
16812025-12-23T22:00:09.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
16822025-12-23T22:00:09.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
16832025-12-23T22:00:09.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16842025-12-23T22:00:09.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
16852025-12-23T22:00:09.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16862025-12-23T22:00:09.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16872025-12-23T22:00:09.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16882025-12-23T22:00:09.799ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
16892025-12-23T22:00:09.799ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 46.660467ms iteration = 17 status = {"blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","inventory_collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
16902025-12-23T22:00:09.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16912025-12-23T22:00:09.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
16922025-12-23T22:00:09.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16932025-12-23T22:00:09.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16942025-12-23T22:00:09.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16952025-12-23T22:00:09.812ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
16962025-12-23T22:00:09.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16972025-12-23T22:00:09.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
16982025-12-23T22:00:09.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
16992025-12-23T22:00:09.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17002025-12-23T22:00:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17012025-12-23T22:00:09.820ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
17022025-12-23T22:00:09.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17032025-12-23T22:00:09.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17042025-12-23T22:00:09.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17052025-12-23T22:00:09.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17062025-12-23T22:00:09.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17072025-12-23T22:00:09.831ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
17082025-12-23T22:00:09.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17092025-12-23T22:00:09.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17102025-12-23T22:00:09.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17112025-12-23T22:00:09.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17122025-12-23T22:00:09.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17132025-12-23T22:00:09.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
17142025-12-23T22:00:09.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17152025-12-23T22:00:09.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17162025-12-23T22:00:09.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17172025-12-23T22:00:09.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17182025-12-23T22:00:09.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17192025-12-23T22:00:09.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17202025-12-23T22:00:09.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17212025-12-23T22:00:09.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
17222025-12-23T22:00:09.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17232025-12-23T22:00:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17242025-12-23T22:00:09.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17252025-12-23T22:00:09.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
17262025-12-23T22:00:09.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17272025-12-23T22:00:09.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17282025-12-23T22:00:09.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17292025-12-23T22:00:09.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17302025-12-23T22:00:09.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17312025-12-23T22:00:09.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17322025-12-23T22:00:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17332025-12-23T22:00:09.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
17342025-12-23T22:00:09.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17352025-12-23T22:00:09.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17362025-12-23T22:00:09.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17372025-12-23T22:00:09.896ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
17382025-12-23T22:00:09.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17392025-12-23T22:00:09.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17402025-12-23T22:00:09.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17412025-12-23T22:00:09.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17422025-12-23T22:00:09.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17432025-12-23T22:00:09.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17442025-12-23T22:00:09.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17452025-12-23T22:00:09.907ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
17462025-12-23T22:00:09.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17472025-12-23T22:00:09.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17482025-12-23T22:00:09.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17492025-12-23T22:00:09.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17502025-12-23T22:00:09.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17512025-12-23T22:00:09.916ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
17522025-12-23T22:00:09.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17532025-12-23T22:00:09.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17542025-12-23T22:00:09.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17552025-12-23T22:00:09.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17562025-12-23T22:00:09.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17572025-12-23T22:00:09.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
17582025-12-23T22:00:09.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17592025-12-23T22:00:09.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17602025-12-23T22:00:09.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17612025-12-23T22:00:09.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17622025-12-23T22:00:09.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17632025-12-23T22:00:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
17642025-12-23T22:00:09.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17652025-12-23T22:00:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17662025-12-23T22:00:09.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17672025-12-23T22:00:09.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17682025-12-23T22:00:09.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17692025-12-23T22:00:09.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
17702025-12-23T22:00:09.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17712025-12-23T22:00:09.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17722025-12-23T22:00:09.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17732025-12-23T22:00:09.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17742025-12-23T22:00:09.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17752025-12-23T22:00:09.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17762025-12-23T22:00:09.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17772025-12-23T22:00:09.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17782025-12-23T22:00:09.956ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
17792025-12-23T22:00:09.956ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
17802025-12-23T22:00:09.956ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 273224e7-df22-4327-9779-561908dc9c82 parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
17812025-12-23T22:00:09.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17822025-12-23T22:00:09.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17832025-12-23T22:00:09.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17842025-12-23T22:00:09.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
17852025-12-23T22:00:09.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17862025-12-23T22:00:09.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17872025-12-23T22:00:09.966ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
17882025-12-23T22:00:09.966ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
17892025-12-23T22:00:09.966ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 213.94318ms iteration = 27 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
17902025-12-23T22:00:11.816ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = be55fb36-d232-4449-adf0-f410c86f3772 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending version = 1.0.0
17912025-12-23T22:00:11.816ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = be55fb36-d232-4449-adf0-f410c86f3772 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
17922025-12-23T22:00:11.816ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = be55fb36-d232-4449-adf0-f410c86f3772 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
17932025-12-23T22:00:11.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2955 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = be55fb36-d232-4449-adf0-f410c86f3772 response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending
17942025-12-23T22:00:11.819ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = f71fb969-b6e9-4bd2-9f2c-3b2c61636ff6 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D version = 1.0.0
17952025-12-23T22:00:11.819ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = f71fb969-b6e9-4bd2-9f2c-3b2c61636ff6 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
17962025-12-23T22:00:11.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = f71fb969-b6e9-4bd2-9f2c-3b2c61636ff6 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
17972025-12-23T22:00:11.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2219 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58500 req_id = f71fb969-b6e9-4bd2-9f2c-3b2c61636ff6 response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D
17982025-12-23T22:00:11.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 470 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = d185019a-ce84-4e91-a6b1-29a4689fb4d4 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
17992025-12-23T22:00:14.754ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
18002025-12-23T22:00:14.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18012025-12-23T22:00:14.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
18022025-12-23T22:00:14.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18032025-12-23T22:00:14.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18042025-12-23T22:00:14.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18052025-12-23T22:00:14.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 123 reason = Timeout
18062025-12-23T22:00:14.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18072025-12-23T22:00:14.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18082025-12-23T22:00:14.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18092025-12-23T22:00:14.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18102025-12-23T22:00:14.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18112025-12-23T22:00:14.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18122025-12-23T22:00:14.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.309501ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18132025-12-23T22:00:14.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18142025-12-23T22:00:14.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18152025-12-23T22:00:14.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18162025-12-23T22:00:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18172025-12-23T22:00:14.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18182025-12-23T22:00:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18192025-12-23T22:00:14.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18202025-12-23T22:00:14.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18212025-12-23T22:00:14.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
18222025-12-23T22:00:14.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.501252ms iteration = 123 status = {"config_updated":false}
18232025-12-23T22:00:14.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242025-12-23T22:00:14.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
18252025-12-23T22:00:14.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18262025-12-23T22:00:14.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18272025-12-23T22:00:14.784ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
18282025-12-23T22:00:14.784ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.13418ms iteration = 62 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
18292025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 56 reason = Timeout
18302025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 42 reason = Timeout
18312025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
18322025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 124 reason = Timeout
18332025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18342025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
18352025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 42 reason = Timeout
18362025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
18372025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
18382025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
18392025-12-23T22:00:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18402025-12-23T22:00:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
18412025-12-23T22:00:19.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
18422025-12-23T22:00:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
18432025-12-23T22:00:19.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18442025-12-23T22:00:19.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18452025-12-23T22:00:19.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18462025-12-23T22:00:19.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18472025-12-23T22:00:19.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
18482025-12-23T22:00:19.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
18492025-12-23T22:00:19.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
18502025-12-23T22:00:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
18512025-12-23T22:00:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
18522025-12-23T22:00:19.763ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
18532025-12-23T22:00:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
18542025-12-23T22:00:19.763ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
18552025-12-23T22:00:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18562025-12-23T22:00:19.763ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18572025-12-23T22:00:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18582025-12-23T22:00:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
18592025-12-23T22:00:19.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.114099ms iteration = 42 status = "NoSitrep"
18602025-12-23T22:00:19.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
18612025-12-23T22:00:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
18622025-12-23T22:00:19.765ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 original_time_started = 2025-12-23 22:00:09.197906 UTC
18632025-12-23T22:00:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.043405ms iteration = 56 status = {"Loaded":{"collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","time_loaded":"2025-12-23T22:00:19.756422538Z","time_started":"2025-12-23T22:00:09.197906Z"}}
18642025-12-23T22:00:19.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
18652025-12-23T22:00:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.311463ms iteration = 124 status = {"config_updated":false}
18662025-12-23T22:00:19.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18672025-12-23T22:00:19.767ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
18682025-12-23T22:00:19.769ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
18692025-12-23T22:00:19.769ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
18702025-12-23T22:00:19.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.497091ms iteration = 42 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
18712025-12-23T22:00:21.823ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 712 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = a795e2c8-db1e-439b-811d-49822f6a8746 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
18722025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
18732025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18742025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
18752025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18762025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18772025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18782025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 125 reason = Timeout
18792025-12-23T22:00:24.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18802025-12-23T22:00:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18812025-12-23T22:00:24.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18822025-12-23T22:00:24.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18832025-12-23T22:00:24.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18842025-12-23T22:00:24.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18852025-12-23T22:00:24.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18862025-12-23T22:00:24.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18872025-12-23T22:00:24.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18882025-12-23T22:00:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18892025-12-23T22:00:24.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18902025-12-23T22:00:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18912025-12-23T22:00:24.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18922025-12-23T22:00:24.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18932025-12-23T22:00:24.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
18942025-12-23T22:00:24.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.633006ms iteration = 125 status = {"config_updated":false}
18952025-12-23T22:00:24.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18962025-12-23T22:00:24.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
18972025-12-23T22:00:24.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18982025-12-23T22:00:24.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18992025-12-23T22:00:24.779ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
19002025-12-23T22:00:24.779ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.889435ms iteration = 63 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
19012025-12-23T22:00:24.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.037982ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19022025-12-23T22:00:29.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 126 reason = Timeout
19032025-12-23T22:00:29.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19042025-12-23T22:00:29.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19052025-12-23T22:00:29.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062025-12-23T22:00:29.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19072025-12-23T22:00:29.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19082025-12-23T22:00:29.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19092025-12-23T22:00:29.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
19102025-12-23T22:00:29.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.572954ms iteration = 126 status = {"config_updated":false}
19112025-12-23T22:00:31.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 507 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 79fc39c9-92d7-45a3-b605-c48a50c03f20 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
19122025-12-23T22:00:34.619ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
19132025-12-23T22:00:34.619ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
19142025-12-23T22:00:34.619ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
19152025-12-23T22:00:34.619ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
19162025-12-23T22:00:34.619ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19172025-12-23T22:00:34.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
19182025-12-23T22:00:34.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
19192025-12-23T22:00:34.622ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 722356578 } })
19202025-12-23T22:00:34.623ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 722834971 } })
19212025-12-23T22:00:34.623ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
19222025-12-23T22:00:34.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 723580132 } })
19232025-12-23T22:00:34.624ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.780505ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19242025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
19252025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
19262025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
19272025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19282025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19292025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
19302025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
19312025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
19322025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
19332025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
19342025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19352025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
19362025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19372025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
19382025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
19392025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19402025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19412025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
19422025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
19432025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19442025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
19452025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
19462025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
19472025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
19482025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
19492025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
19502025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 33 reason = Timeout
19512025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19522025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
19532025-12-23T22:00:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19542025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
19552025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
19562025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19572025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.31\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"sps":[]}
19582025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 43 reason = Timeout
19592025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
19602025-12-23T22:00:34.756ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19612025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 43 reason = Timeout
19622025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
19632025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
19642025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19652025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19662025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19672025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 127 reason = Timeout
19682025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
19692025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19702025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 57 reason = Timeout
19712025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19722025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19732025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19742025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
19752025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19762025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
19772025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
19782025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19792025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19802025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
19812025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19822025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19832025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19842025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
19852025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
19862025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
19872025-12-23T22:00:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19882025-12-23T22:00:34.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19892025-12-23T22:00:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19902025-12-23T22:00:34.758ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19912025-12-23T22:00:34.758ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
19922025-12-23T22:00:34.759ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19932025-12-23T22:00:34.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.656123ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
19942025-12-23T22:00:34.759ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19952025-12-23T22:00:34.759ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19962025-12-23T22:00:34.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19972025-12-23T22:00:34.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19982025-12-23T22:00:34.760ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19992025-12-23T22:00:34.761ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 141.266937ms iteration = 22 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
20002025-12-23T22:00:34.761ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20012025-12-23T22:00:34.761ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20022025-12-23T22:00:34.761ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20032025-12-23T22:00:34.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
20042025-12-23T22:00:34.762ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20052025-12-23T22:00:34.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20062025-12-23T22:00:34.764ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20072025-12-23T22:00:34.764ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20082025-12-23T22:00:34.764ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20092025-12-23T22:00:34.764ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20102025-12-23T22:00:34.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20112025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
20122025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20132025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20142025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20152025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20162025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
20172025-12-23T22:00:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20182025-12-23T22:00:34.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
20192025-12-23T22:00:34.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
20202025-12-23T22:00:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
20212025-12-23T22:00:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20222025-12-23T22:00:34.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20232025-12-23T22:00:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20242025-12-23T22:00:34.768ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20252025-12-23T22:00:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
20262025-12-23T22:00:34.768ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20272025-12-23T22:00:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20282025-12-23T22:00:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20292025-12-23T22:00:34.771ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.30106ms iteration = 22 status = 0
20302025-12-23T22:00:34.774ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20312025-12-23T22:00:34.774ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
20322025-12-23T22:00:34.774ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.518573ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
20332025-12-23T22:00:34.776ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 20.360045ms iteration = 43 status = "NoSitrep"
20342025-12-23T22:00:34.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20352025-12-23T22:00:34.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20362025-12-23T22:00:34.778ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20372025-12-23T22:00:34.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20382025-12-23T22:00:34.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20392025-12-23T22:00:34.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20402025-12-23T22:00:34.782ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
20412025-12-23T22:00:34.782ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20422025-12-23T22:00:34.782ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fc2eb9c-59c8-423c-9536-54d06816549d", "content-length": "26", "date": "Tue, 23 Dec 2025 22:00:34 GMT"} })
20432025-12-23T22:00:34.782ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20442025-12-23T22:00:34.784ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20452025-12-23T22:00:34.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
20462025-12-23T22:00:34.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20472025-12-23T22:00:34.785ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 885191143 } })
20482025-12-23T22:00:34.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20492025-12-23T22:00:34.809ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 53.814149ms iteration = 36 status = {"physical_disks_added":0}
20502025-12-23T22:00:34.818ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
20512025-12-23T22:00:34.818ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 63.518583ms iteration = 64 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
20522025-12-23T22:00:34.822ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20532025-12-23T22:00:34.824ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
20542025-12-23T22:00:34.829ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
20552025-12-23T22:00:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
20562025-12-23T22:00:34.830ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
20572025-12-23T22:00:34.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
20582025-12-23T22:00:34.833ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20592025-12-23T22:00:34.834ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20602025-12-23T22:00:34.835ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20612025-12-23T22:00:34.835ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20622025-12-23T22:00:34.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20632025-12-23T22:00:34.836ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
20642025-12-23T22:00:34.837ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20652025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
20662025-12-23T22:00:34.838ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
20672025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 82.257312ms iteration = 43 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
20682025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20692025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20702025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20712025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20722025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20732025-12-23T22:00:34.838ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 219.032003ms iteration = 22 status = {}
20742025-12-23T22:00:34.839ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 84.062023ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
20752025-12-23T22:00:34.840ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20762025-12-23T22:00:34.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20772025-12-23T22:00:34.840ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20782025-12-23T22:00:34.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20792025-12-23T22:00:34.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20802025-12-23T22:00:34.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20812025-12-23T22:00:34.843ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20822025-12-23T22:00:34.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20832025-12-23T22:00:34.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
20842025-12-23T22:00:34.844ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
20852025-12-23T22:00:34.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
20862025-12-23T22:00:34.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20872025-12-23T22:00:34.845ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20882025-12-23T22:00:34.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20892025-12-23T22:00:34.846ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20902025-12-23T22:00:34.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20912025-12-23T22:00:34.848ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20922025-12-23T22:00:34.849ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 948911212 } })
20932025-12-23T22:00:34.849ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
20942025-12-23T22:00:34.850ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.156389ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
20952025-12-23T22:00:34.850ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 949856489 } })
20962025-12-23T22:00:34.851ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
20972025-12-23T22:00:34.851ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 94.909369ms iteration = 33 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
20982025-12-23T22:00:34.851ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
20992025-12-23T22:00:34.851ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 95.704912ms iteration = 127 status = {"config_updated":false}
21002025-12-23T22:00:34.852ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21012025-12-23T22:00:34.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21022025-12-23T22:00:34.853ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21032025-12-23T22:00:34.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
21042025-12-23T22:00:34.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
21052025-12-23T22:00:34.854ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
21062025-12-23T22:00:34.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
21072025-12-23T22:00:34.857ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 original_time_started = 2025-12-23 22:00:09.197906 UTC
21082025-12-23T22:00:34.857ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 101.208167ms iteration = 57 status = {"Loaded":{"collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","time_loaded":"2025-12-23T22:00:34.756048347Z","time_started":"2025-12-23T22:00:09.197906Z"}}
21092025-12-23T22:00:34.859ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21102025-12-23T22:00:34.859ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21112025-12-23T22:00:34.859ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 104.015927ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21122025-12-23T22:00:34.900ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 144.662563ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21132025-12-23T22:00:34.910ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 154.935974ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21142025-12-23T22:00:34.915ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21152025-12-23T22:00:34.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21162025-12-23T22:00:34.916ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 161.482488ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21172025-12-23T22:00:34.921ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
21182025-12-23T22:00:34.921ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21192025-12-23T22:00:34.921ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "820d821f-9d42-4ed4-a1fd-5bf8accc79ad", "content-length": "26", "date": "Tue, 23 Dec 2025 22:00:34 GMT"} })
21202025-12-23T22:00:34.921ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21212025-12-23T22:00:34.923ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
21222025-12-23T22:00:34.950ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
21232025-12-23T22:00:34.950ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21242025-12-23T22:00:34.950ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6394563c-ceda-439d-b572-f4ca52462c9c", "content-length": "372", "date": "Tue, 23 Dec 2025 22:00:34 GMT"} })
21252025-12-23T22:00:34.950ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21262025-12-23T22:00:34.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21272025-12-23T22:00:34.966ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
21282025-12-23T22:00:34.966ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
21292025-12-23T22:00:34.967ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "63130af6-897e-491f-a496-db88d722c978", "date": "Tue, 23 Dec 2025 22:00:34 GMT"} })
21302025-12-23T22:00:34.967ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21312025-12-23T22:00:34.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21322025-12-23T22:00:34.969ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21332025-12-23T22:00:34.969ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 213.678952ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
21342025-12-23T22:00:34.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21352025-12-23T22:00:34.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21362025-12-23T22:00:34.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21372025-12-23T22:00:34.974ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21382025-12-23T22:00:34.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21392025-12-23T22:00:34.975ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1533, tv_nsec: 75458554 } })
21402025-12-23T22:00:34.977ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
21412025-12-23T22:00:34.977ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21422025-12-23T22:00:34.978ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b80b47ce-78a3-478e-9384-908057c92b05", "date": "Tue, 23 Dec 2025 22:00:34 GMT"} })
21432025-12-23T22:00:34.979ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 223.380326ms iteration = 33 status = {}
21442025-12-23T22:00:34.982ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21452025-12-23T22:00:34.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21462025-12-23T22:00:34.984ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21472025-12-23T22:00:34.986ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21482025-12-23T22:00:34.986ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21492025-12-23T22:00:34.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21502025-12-23T22:00:34.988ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21512025-12-23T22:00:34.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21522025-12-23T22:00:34.991ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21532025-12-23T22:00:34.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21542025-12-23T22:00:35.027ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1533, tv_nsec: 127592096 } })
21552025-12-23T22:00:35.057ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
21562025-12-23T22:00:35.061ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd4ee436-09e4-413e-a8b3-a169d5c5be51", "content-length": "691", "date": "Tue, 23 Dec 2025 22:00:35 GMT"} })
21572025-12-23T22:00:35.061ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
21582025-12-23T22:00:35.061ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
21592025-12-23T22:00:35.061ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
21602025-12-23T22:00:35.063ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 307.51309ms iteration = 22 status = {}
21612025-12-23T22:00:35.076ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21622025-12-23T22:00:35.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21632025-12-23T22:00:35.079ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
21642025-12-23T22:00:35.079ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21652025-12-23T22:00:35.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21662025-12-23T22:00:35.093ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
21672025-12-23T22:00:35.093ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
21682025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
21692025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
21702025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
21712025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
21722025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
21732025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
21742025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
21752025-12-23T22:00:35.094ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
21762025-12-23T22:00:35.095ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
21772025-12-23T22:00:35.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21782025-12-23T22:00:35.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21792025-12-23T22:00:35.106ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
21802025-12-23T22:00:35.106ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
21812025-12-23T22:00:35.366ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a738cb9f-b029-4c01-be02-75d7f583bc17", "date": "Tue, 23 Dec 2025 22:00:35 GMT"} })
21822025-12-23T22:00:35.366ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
21832025-12-23T22:00:35.367ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
21842025-12-23T22:00:35.367ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "affcaeed-9452-46b0-b838-646d5b80d049", "content-length": "682", "date": "Tue, 23 Dec 2025 22:00:35 GMT"} })
21852025-12-23T22:00:35.367ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21862025-12-23T22:00:35.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21872025-12-23T22:00:35.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21882025-12-23T22:00:35.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21892025-12-23T22:00:35.373ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21902025-12-23T22:00:35.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21912025-12-23T22:00:35.375ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21922025-12-23T22:00:35.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21932025-12-23T22:00:35.382ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 626.009121ms iteration = 22 status = {}
21942025-12-23T22:00:39.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 128 reason = Timeout
21952025-12-23T22:00:39.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
21962025-12-23T22:00:39.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
21972025-12-23T22:00:39.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21982025-12-23T22:00:39.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
21992025-12-23T22:00:39.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22002025-12-23T22:00:39.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22012025-12-23T22:00:39.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
22022025-12-23T22:00:39.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.127442ms iteration = 128 status = {"config_updated":false}
22032025-12-23T22:00:41.823ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 526 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = b3cf8b10-0bb5-4b1d-9364-8043aeae66e9 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
22042025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
22052025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
22062025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22072025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22082025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22092025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22102025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 129 reason = Timeout
22112025-12-23T22:00:44.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22122025-12-23T22:00:44.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22132025-12-23T22:00:44.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22142025-12-23T22:00:44.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22152025-12-23T22:00:44.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22162025-12-23T22:00:44.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22172025-12-23T22:00:44.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.632111ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22182025-12-23T22:00:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22192025-12-23T22:00:44.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22202025-12-23T22:00:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22212025-12-23T22:00:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
22222025-12-23T22:00:44.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22232025-12-23T22:00:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22242025-12-23T22:00:44.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
22252025-12-23T22:00:44.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.283204ms iteration = 129 status = {"config_updated":false}
22262025-12-23T22:00:44.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22272025-12-23T22:00:44.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22282025-12-23T22:00:44.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22292025-12-23T22:00:44.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
22302025-12-23T22:00:44.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22312025-12-23T22:00:44.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22322025-12-23T22:00:44.787ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
22332025-12-23T22:00:44.787ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.128757ms iteration = 65 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
22342025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 44 reason = Timeout
22352025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 58 reason = Timeout
22362025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
22372025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
22382025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 44 reason = Timeout
22392025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 130 reason = Timeout
22402025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22412025-12-23T22:00:49.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22422025-12-23T22:00:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
22432025-12-23T22:00:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
22442025-12-23T22:00:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22452025-12-23T22:00:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22462025-12-23T22:00:49.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22472025-12-23T22:00:49.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22482025-12-23T22:00:49.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22492025-12-23T22:00:49.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22502025-12-23T22:00:49.759ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22512025-12-23T22:00:49.760ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22522025-12-23T22:00:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
22532025-12-23T22:00:49.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
22542025-12-23T22:00:49.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
22552025-12-23T22:00:49.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
22562025-12-23T22:00:49.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
22572025-12-23T22:00:49.764ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
22582025-12-23T22:00:49.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
22592025-12-23T22:00:49.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
22602025-12-23T22:00:49.765ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22612025-12-23T22:00:49.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22622025-12-23T22:00:49.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22632025-12-23T22:00:49.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22642025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.62678ms iteration = 44 status = "NoSitrep"
22652025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 original_time_started = 2025-12-23 22:00:09.197906 UTC
22662025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 10.690792ms iteration = 58 status = {"Loaded":{"collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","time_loaded":"2025-12-23T22:00:49.756427408Z","time_started":"2025-12-23T22:00:09.197906Z"}}
22672025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22682025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22692025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
22702025-12-23T22:00:49.767ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.446294ms iteration = 130 status = {"config_updated":false}
22712025-12-23T22:00:49.769ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22722025-12-23T22:00:49.771ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
22732025-12-23T22:00:49.774ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
22742025-12-23T22:00:49.774ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
22752025-12-23T22:00:49.774ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 17.944836ms iteration = 44 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
22762025-12-23T22:00:51.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 443 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = e016acdb-71d0-4005-907d-44086bcfbe68 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
22772025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
22782025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22792025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
22802025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22812025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22822025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 131 reason = Timeout
22832025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22842025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22852025-12-23T22:00:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22862025-12-23T22:00:54.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22872025-12-23T22:00:54.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22882025-12-23T22:00:54.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22892025-12-23T22:00:54.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22902025-12-23T22:00:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
22912025-12-23T22:00:54.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22922025-12-23T22:00:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22932025-12-23T22:00:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22942025-12-23T22:00:54.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22952025-12-23T22:00:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22962025-12-23T22:00:54.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
22972025-12-23T22:00:54.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.399259ms iteration = 131 status = {"config_updated":false}
22982025-12-23T22:00:54.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22992025-12-23T22:00:54.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23002025-12-23T22:00:54.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23012025-12-23T22:00:54.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
23022025-12-23T22:00:54.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23032025-12-23T22:00:54.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23042025-12-23T22:00:54.780ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
23052025-12-23T22:00:54.780ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.546001ms iteration = 66 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
23062025-12-23T22:00:54.795ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.426371ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23072025-12-23T22:00:59.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 132 reason = Timeout
23082025-12-23T22:00:59.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23092025-12-23T22:00:59.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23102025-12-23T22:00:59.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23112025-12-23T22:00:59.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
23122025-12-23T22:00:59.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23132025-12-23T22:00:59.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23142025-12-23T22:00:59.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
23152025-12-23T22:00:59.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.618085ms iteration = 132 status = {"config_updated":false}
23162025-12-23T22:01:01.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 504 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 675b93cb-3bcc-4582-9c2a-31d932c02d8f response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
23172025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
23182025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
23192025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
23202025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
23212025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23222025-12-23T22:01:04.619ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a98c752e-5a58-41fa-9f53-a8d4f529a42f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e9c3ce58-81e4-4140-937f-1b169b78f235.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 719330272 } })
23232025-12-23T22:01:04.619ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2bc6e36d-0fab-4746-9740-073e15922b00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 719343902 } })
23242025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
23252025-12-23T22:01:04.619ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-23 21:51:04.619721560 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
23262025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23272025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
23282025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
23292025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
23302025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
23312025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
23322025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
23332025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
23342025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23352025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
23362025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
23372025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
23382025-12-23T22:01:04.619ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
23392025-12-23T22:01:04.619ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = ExternalNetworking step id = Ensure
23402025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
23412025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23422025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
23432025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
23442025-12-23T22:01:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23452025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
23462025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
23472025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23482025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.312027ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
23492025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
23502025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
23512025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
23522025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23532025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
23542025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.553824ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
23552025-12-23T22:01:04.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23562025-12-23T22:01:04.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23572025-12-23T22:01:04.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23582025-12-23T22:01:04.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23592025-12-23T22:01:04.622ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23602025-12-23T22:01:04.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23612025-12-23T22:01:04.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 724018184 } })
23622025-12-23T22:01:04.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23632025-12-23T22:01:04.624ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-23 21:51:04.619721560 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
23642025-12-23T22:01:04.624ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.860568ms iteration = 12 status = {"expiration":"2025-12-23T21:51:04.619721560Z","pruned":[]}
23652025-12-23T22:01:04.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23662025-12-23T22:01:04.624ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 724366844 } })
23672025-12-23T22:01:04.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23682025-12-23T22:01:04.625ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
23692025-12-23T22:01:04.625ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 725153806 } })
23702025-12-23T22:01:04.627ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.618875ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
23712025-12-23T22:01:04.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23722025-12-23T22:01:04.627ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23732025-12-23T22:01:04.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23742025-12-23T22:01:04.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
23752025-12-23T22:01:04.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23762025-12-23T22:01:04.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23772025-12-23T22:01:04.631ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23782025-12-23T22:01:04.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23792025-12-23T22:01:04.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
23802025-12-23T22:01:04.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23812025-12-23T22:01:04.634ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23822025-12-23T22:01:04.635ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
23832025-12-23T22:01:04.635ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
23842025-12-23T22:01:04.635ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
23852025-12-23T22:01:04.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23862025-12-23T22:01:04.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23872025-12-23T22:01:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
23882025-12-23T22:01:04.639ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
23892025-12-23T22:01:04.639ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
23902025-12-23T22:01:04.639ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
23912025-12-23T22:01:04.639ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23922025-12-23T22:01:04.639ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23932025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23942025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
23952025-12-23T22:01:04.640ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
23962025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 28 reason = Timeout
23972025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
23982025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
23992025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
24002025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
24012025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
24022025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
24032025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
24042025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
24052025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 40 reason = Timeout
24062025-12-23T22:01:04.640ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
24072025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 12 reason = Timeout
24082025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.94\u{b5}s iteration = 40 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
24092025-12-23T22:01:04.640ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
24102025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
24112025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
24122025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
24132025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
24142025-12-23T22:01:04.640ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
24152025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
24162025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24172025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
24182025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
24192025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
24202025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
24212025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
24222025-12-23T22:01:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
24232025-12-23T22:01:04.643ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
24242025-12-23T22:01:04.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24252025-12-23T22:01:04.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24262025-12-23T22:01:04.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
24272025-12-23T22:01:04.647ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24282025-12-23T22:01:04.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24292025-12-23T22:01:04.647ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.285885ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
24302025-12-23T22:01:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
24312025-12-23T22:01:04.649ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24322025-12-23T22:01:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
24332025-12-23T22:01:04.650ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
24342025-12-23T22:01:04.650ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 10.070905ms iteration = 12 status = {"by_rx":{},"error":null}
24352025-12-23T22:01:04.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
24362025-12-23T22:01:04.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24372025-12-23T22:01:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24382025-12-23T22:01:04.652ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
24392025-12-23T22:01:04.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24402025-12-23T22:01:04.652ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24412025-12-23T22:01:04.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24422025-12-23T22:01:04.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24432025-12-23T22:01:04.664ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24442025-12-23T22:01:04.664ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24452025-12-23T22:01:04.664ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
24462025-12-23T22:01:04.666ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
24472025-12-23T22:01:04.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
24482025-12-23T22:01:04.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
24492025-12-23T22:01:04.670ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
24502025-12-23T22:01:04.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
24512025-12-23T22:01:04.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
24522025-12-23T22:01:04.670ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
24532025-12-23T22:01:04.671ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
24542025-12-23T22:01:04.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
24552025-12-23T22:01:04.673ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24562025-12-23T22:01:04.673ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
24572025-12-23T22:01:04.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24582025-12-23T22:01:04.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24592025-12-23T22:01:04.674ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
24602025-12-23T22:01:04.675ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
24612025-12-23T22:01:04.675ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
24622025-12-23T22:01:04.675ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 35.533413ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
24632025-12-23T22:01:04.676ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
24642025-12-23T22:01:04.676ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 36.039848ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"215.0.0","globs_reprocessed":{},"no_receivers":[]}
24652025-12-23T22:01:04.677ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
24662025-12-23T22:01:04.677ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
24672025-12-23T22:01:04.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
24682025-12-23T22:01:04.679ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
24692025-12-23T22:01:04.681ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
24702025-12-23T22:01:04.681ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
24712025-12-23T22:01:04.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
24722025-12-23T22:01:04.683ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
24732025-12-23T22:01:04.684ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24742025-12-23T22:01:04.685ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
24752025-12-23T22:01:04.688ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
24762025-12-23T22:01:04.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
24772025-12-23T22:01:04.690ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
24782025-12-23T22:01:04.690ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
24792025-12-23T22:01:04.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
24802025-12-23T22:01:04.690ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = SledList step id = Fetch
24812025-12-23T22:01:04.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24822025-12-23T22:01:04.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24832025-12-23T22:01:04.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24842025-12-23T22:01:04.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
24852025-12-23T22:01:04.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24862025-12-23T22:01:04.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24872025-12-23T22:01:04.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24882025-12-23T22:01:04.709ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
24892025-12-23T22:01:04.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24902025-12-23T22:01:04.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24912025-12-23T22:01:04.713ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = DeployNexusRecords step id = Ensure
24922025-12-23T22:01:04.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24932025-12-23T22:01:04.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24942025-12-23T22:01:04.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24952025-12-23T22:01:04.721ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
24962025-12-23T22:01:04.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24972025-12-23T22:01:04.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24982025-12-23T22:01:04.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24992025-12-23T22:01:04.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25002025-12-23T22:01:04.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25012025-12-23T22:01:04.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
25022025-12-23T22:01:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
25032025-12-23T22:01:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
25042025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
25052025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
25062025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25072025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25082025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
25092025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
25102025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25112025-12-23T22:01:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
25122025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
25132025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25142025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
25152025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
25162025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25172025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25182025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
25192025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
25202025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
25212025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
25222025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25232025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 20.971\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
25242025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25252025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
25262025-12-23T22:01:04.757ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
25272025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25282025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 45 reason = Timeout
25292025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 34 reason = Timeout
25302025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25312025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
25322025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25332025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
25342025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25352025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25362025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
25372025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25382025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25392025-12-23T22:01:04.757ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
25402025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
25412025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
25422025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
25432025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
25442025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25452025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 45 reason = Timeout
25462025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 59 reason = Timeout
25472025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
25482025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
25492025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 133 reason = Timeout
25502025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
25512025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25522025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25532025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Timeout
25542025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
25552025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25562025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25572025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
25582025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
25592025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25602025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25612025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
25622025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
25632025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25642025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25652025-12-23T22:01:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
25662025-12-23T22:01:04.760ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = SledAgent step id = Ensure
25672025-12-23T22:01:04.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25682025-12-23T22:01:04.762ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25692025-12-23T22:01:04.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25702025-12-23T22:01:04.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
25712025-12-23T22:01:04.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
25722025-12-23T22:01:04.765ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
25732025-12-23T22:01:04.765ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
25742025-12-23T22:01:04.765ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
25752025-12-23T22:01:04.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25762025-12-23T22:01:04.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25772025-12-23T22:01:04.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
25782025-12-23T22:01:04.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25792025-12-23T22:01:04.767ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25802025-12-23T22:01:04.768ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25812025-12-23T22:01:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25822025-12-23T22:01:04.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25832025-12-23T22:01:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25842025-12-23T22:01:04.769ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25852025-12-23T22:01:04.769ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25862025-12-23T22:01:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25872025-12-23T22:01:04.772ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25882025-12-23T22:01:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25892025-12-23T22:01:04.773ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
25902025-12-23T22:01:04.773ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
25912025-12-23T22:01:04.773ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25922025-12-23T22:01:04.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25932025-12-23T22:01:04.773ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25942025-12-23T22:01:04.775ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25952025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25962025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25972025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
25982025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25992025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
26002025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26012025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26022025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
26032025-12-23T22:01:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26042025-12-23T22:01:04.777ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26052025-12-23T22:01:04.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
26062025-12-23T22:01:04.777ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
26072025-12-23T22:01:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26082025-12-23T22:01:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
26092025-12-23T22:01:04.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26102025-12-23T22:01:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26112025-12-23T22:01:04.778ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26122025-12-23T22:01:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
26132025-12-23T22:01:04.779ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 879344028 } })
26142025-12-23T22:01:04.780ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
26152025-12-23T22:01:04.780ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
26162025-12-23T22:01:04.780ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
26172025-12-23T22:01:04.780ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 139.626441ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
26182025-12-23T22:01:04.780ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1562, tv_nsec: 880345817 } })
26192025-12-23T22:01:04.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
26202025-12-23T22:01:04.781ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 23.75138ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
26212025-12-23T22:01:04.781ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26222025-12-23T22:01:04.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26232025-12-23T22:01:04.783ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26242025-12-23T22:01:04.783ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26252025-12-23T22:01:04.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26262025-12-23T22:01:04.784ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26272025-12-23T22:01:04.785ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
26282025-12-23T22:01:04.785ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 27.086205ms iteration = 133 status = {"config_updated":false}
26292025-12-23T22:01:04.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26302025-12-23T22:01:04.786ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 original_time_started = 2025-12-23 22:00:09.197906 UTC
26312025-12-23T22:01:04.786ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 28.32331ms iteration = 59 status = {"Loaded":{"collection_id":"f92b2309-ea11-4dbe-892d-2a47b29ba690","time_loaded":"2025-12-23T22:01:04.758052424Z","time_started":"2025-12-23T22:00:09.197906Z"}}
26322025-12-23T22:01:04.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
26332025-12-23T22:01:04.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26342025-12-23T22:01:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26352025-12-23T22:01:04.787ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26362025-12-23T22:01:04.787ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26372025-12-23T22:01:04.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26382025-12-23T22:01:04.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26392025-12-23T22:01:04.788ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.258991ms iteration = 23 status = 0
26402025-12-23T22:01:04.789ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
26412025-12-23T22:01:04.789ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 31.924341ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
26422025-12-23T22:01:04.791ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
26432025-12-23T22:01:04.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26442025-12-23T22:01:04.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26452025-12-23T22:01:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26462025-12-23T22:01:04.793ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26472025-12-23T22:01:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26482025-12-23T22:01:04.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
26492025-12-23T22:01:04.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26502025-12-23T22:01:04.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26512025-12-23T22:01:04.795ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
26522025-12-23T22:01:04.795ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
26532025-12-23T22:01:04.795ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-23 21:50:07.200941 UTC dns_group = external generation = 2
26542025-12-23T22:01:04.795ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 175.73081ms iteration = 12 status = {"generation":2}
26552025-12-23T22:01:04.808ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
26562025-12-23T22:01:04.808ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
26572025-12-23T22:01:04.808ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-23 21:50:06.190854 UTC dns_group = internal generation = 1
26582025-12-23T22:01:04.808ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 189.249511ms iteration = 12 status = {"generation":1}
26592025-12-23T22:01:04.810ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26602025-12-23T22:01:04.813ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
26612025-12-23T22:01:04.832ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
26622025-12-23T22:01:04.832ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
26632025-12-23T22:01:04.832ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 74.164744ms iteration = 45 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
26642025-12-23T22:01:04.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26652025-12-23T22:01:04.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
26662025-12-23T22:01:04.842ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
26672025-12-23T22:01:04.842ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 85.37386ms iteration = 67 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
26682025-12-23T22:01:04.861ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26692025-12-23T22:01:04.865ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26702025-12-23T22:01:04.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
26712025-12-23T22:01:04.866ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
26722025-12-23T22:01:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
26732025-12-23T22:01:04.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26742025-12-23T22:01:04.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26752025-12-23T22:01:04.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 115.682346ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26762025-12-23T22:01:04.874ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26772025-12-23T22:01:04.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26782025-12-23T22:01:04.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26792025-12-23T22:01:04.877ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 120.041929ms iteration = 45 status = "NoSitrep"
26802025-12-23T22:01:04.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26812025-12-23T22:01:04.887ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 130.082602ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
26822025-12-23T22:01:04.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26832025-12-23T22:01:04.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26842025-12-23T22:01:04.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26852025-12-23T22:01:04.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26862025-12-23T22:01:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26872025-12-23T22:01:04.903ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26882025-12-23T22:01:04.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26892025-12-23T22:01:04.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
26902025-12-23T22:01:04.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26912025-12-23T22:01:04.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26922025-12-23T22:01:04.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
26932025-12-23T22:01:04.910ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26942025-12-23T22:01:04.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26952025-12-23T22:01:04.913ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26962025-12-23T22:01:04.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26972025-12-23T22:01:04.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26982025-12-23T22:01:04.918ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 160.212493ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
26992025-12-23T22:01:04.919ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
27002025-12-23T22:01:04.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27012025-12-23T22:01:04.919ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27022025-12-23T22:01:04.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27032025-12-23T22:01:04.920ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27042025-12-23T22:01:04.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27052025-12-23T22:01:04.923ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 166.088758ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27062025-12-23T22:01:04.924ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27072025-12-23T22:01:04.924ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27082025-12-23T22:01:04.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27092025-12-23T22:01:04.925ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27102025-12-23T22:01:04.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
27112025-12-23T22:01:04.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27122025-12-23T22:01:04.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27132025-12-23T22:01:04.927ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 169.382462ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
27142025-12-23T22:01:04.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27152025-12-23T22:01:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27162025-12-23T22:01:04.932ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27172025-12-23T22:01:04.932ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
27182025-12-23T22:01:04.932ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27192025-12-23T22:01:04.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
27202025-12-23T22:01:04.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27212025-12-23T22:01:04.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
27222025-12-23T22:01:04.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27232025-12-23T22:01:04.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27242025-12-23T22:01:04.946ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27252025-12-23T22:01:04.949ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
27262025-12-23T22:01:04.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27272025-12-23T22:01:04.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27282025-12-23T22:01:04.950ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
27292025-12-23T22:01:04.950ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27302025-12-23T22:01:04.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
27312025-12-23T22:01:04.951ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 332.130525ms iteration = 23 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
27322025-12-23T22:01:04.954ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 199.236164ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
27332025-12-23T22:01:04.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27342025-12-23T22:01:04.957ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
27352025-12-23T22:01:04.962ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
27362025-12-23T22:01:04.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27372025-12-23T22:01:04.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27382025-12-23T22:01:04.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27392025-12-23T22:01:04.971ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
27402025-12-23T22:01:04.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27412025-12-23T22:01:04.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27422025-12-23T22:01:04.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27432025-12-23T22:01:04.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27442025-12-23T22:01:04.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27452025-12-23T22:01:04.984ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
27462025-12-23T22:01:04.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27472025-12-23T22:01:04.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27482025-12-23T22:01:04.990ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
27492025-12-23T22:01:04.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
27502025-12-23T22:01:04.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27512025-12-23T22:01:04.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27522025-12-23T22:01:04.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27532025-12-23T22:01:04.997ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
27542025-12-23T22:01:05.003ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
27552025-12-23T22:01:05.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27562025-12-23T22:01:05.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27572025-12-23T22:01:05.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27582025-12-23T22:01:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27592025-12-23T22:01:05.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27602025-12-23T22:01:05.009ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
27612025-12-23T22:01:05.009ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 251.819599ms iteration = 34 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
27622025-12-23T22:01:05.012ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
27632025-12-23T22:01:05.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27642025-12-23T22:01:05.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27652025-12-23T22:01:05.027ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27662025-12-23T22:01:05.027ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
27672025-12-23T22:01:05.027ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27682025-12-23T22:01:05.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
27692025-12-23T22:01:05.027ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
27702025-12-23T22:01:05.028ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27712025-12-23T22:01:05.028ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7db6258b-9ac5-437a-99c0-cdf5404b32ac", "content-length": "26", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
27722025-12-23T22:01:05.028ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27732025-12-23T22:01:05.030ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27742025-12-23T22:01:05.031ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1563, tv_nsec: 130988092 } })
27752025-12-23T22:01:05.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27762025-12-23T22:01:05.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27772025-12-23T22:01:05.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27782025-12-23T22:01:05.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27792025-12-23T22:01:05.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27802025-12-23T22:01:05.052ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
27812025-12-23T22:01:05.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27822025-12-23T22:01:05.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27832025-12-23T22:01:05.052ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-23T21:50:00.171530Z, time_modified: 2025-12-23T21:50:00.171530Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }) }
27842025-12-23T22:01:05.052ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 433.002443ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]}},"default_endpoint":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
27852025-12-23T22:01:05.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27862025-12-23T22:01:05.059ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
27872025-12-23T22:01:05.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27882025-12-23T22:01:05.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27892025-12-23T22:01:05.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27902025-12-23T22:01:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27912025-12-23T22:01:05.081ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
27922025-12-23T22:01:05.083ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "9f24d264-6d1a-4187-8e6b-2d2107ef52da", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
27932025-12-23T22:01:05.083ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
27942025-12-23T22:01:05.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27952025-12-23T22:01:05.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27962025-12-23T22:01:05.085ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
27972025-12-23T22:01:05.085ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 445.227379ms iteration = 12 status = {"errors":[],"probes_by_sled":{"ff85f534-787c-483e-ba89-a37cdea1dd38":0}}
27982025-12-23T22:01:05.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27992025-12-23T22:01:05.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
28002025-12-23T22:01:05.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28012025-12-23T22:01:05.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28022025-12-23T22:01:05.094ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
28032025-12-23T22:01:05.094ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28042025-12-23T22:01:05.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28052025-12-23T22:01:05.094ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "28f30b99-51a9-46c6-ae04-77e2bfea691c", "content-length": "26", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
28062025-12-23T22:01:05.094ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28072025-12-23T22:01:05.097ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
28082025-12-23T22:01:05.100ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
28092025-12-23T22:01:05.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28102025-12-23T22:01:05.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28112025-12-23T22:01:05.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28122025-12-23T22:01:05.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28132025-12-23T22:01:05.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28142025-12-23T22:01:05.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28152025-12-23T22:01:05.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28162025-12-23T22:01:05.113ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28172025-12-23T22:01:05.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
28182025-12-23T22:01:05.113ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 355.573877ms iteration = 37 status = {"physical_disks_added":0}
28192025-12-23T22:01:05.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
28202025-12-23T22:01:05.117ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28212025-12-23T22:01:05.117ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
28222025-12-23T22:01:05.117ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 360.645751ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
28232025-12-23T22:01:05.130ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
28242025-12-23T22:01:05.132ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "09522aeb-ee92-4e95-9f6e-64809c1e8f16", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} }) sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
28252025-12-23T22:01:05.134ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = FirewallRules step id = Ensure
28262025-12-23T22:01:05.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28272025-12-23T22:01:05.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28282025-12-23T22:01:05.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28292025-12-23T22:01:05.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28302025-12-23T22:01:05.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28312025-12-23T22:01:05.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28322025-12-23T22:01:05.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28332025-12-23T22:01:05.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28342025-12-23T22:01:05.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28352025-12-23T22:01:05.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28362025-12-23T22:01:05.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28372025-12-23T22:01:05.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28382025-12-23T22:01:05.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28392025-12-23T22:01:05.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28402025-12-23T22:01:05.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28412025-12-23T22:01:05.195ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
28422025-12-23T22:01:05.195ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c5d1605f-7b7f-4b75-aa62-830280fdb109", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28432025-12-23T22:01:05.195ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28442025-12-23T22:01:05.198ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "292b80e1-8302-445e-9168-23e6944b2577", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28452025-12-23T22:01:05.198ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28462025-12-23T22:01:05.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
28472025-12-23T22:01:05.200ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28482025-12-23T22:01:05.200ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 580.370123ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
28492025-12-23T22:01:05.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28502025-12-23T22:01:05.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28512025-12-23T22:01:05.235ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
28522025-12-23T22:01:05.235ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28532025-12-23T22:01:05.235ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3ae9f42-0e3c-467e-966c-7f46676665b2", "content-length": "372", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
28542025-12-23T22:01:05.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28552025-12-23T22:01:05.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28562025-12-23T22:01:05.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28572025-12-23T22:01:05.241ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
28582025-12-23T22:01:05.242ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
28592025-12-23T22:01:05.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28602025-12-23T22:01:05.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28612025-12-23T22:01:05.244ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 624.846988ms iteration = 23 status = {}
28622025-12-23T22:01:05.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28632025-12-23T22:01:05.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28642025-12-23T22:01:05.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28652025-12-23T22:01:05.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28662025-12-23T22:01:05.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28672025-12-23T22:01:05.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28682025-12-23T22:01:05.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28692025-12-23T22:01:05.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28702025-12-23T22:01:05.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28712025-12-23T22:01:05.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
28722025-12-23T22:01:05.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28732025-12-23T22:01:05.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28742025-12-23T22:01:05.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28752025-12-23T22:01:05.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28762025-12-23T22:01:05.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28772025-12-23T22:01:05.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28782025-12-23T22:01:05.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28792025-12-23T22:01:05.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28802025-12-23T22:01:05.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28812025-12-23T22:01:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28822025-12-23T22:01:05.272ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1563, tv_nsec: 371792119 } })
28832025-12-23T22:01:05.296ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
28842025-12-23T22:01:05.296ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 656.024678ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
28852025-12-23T22:01:05.313ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
28862025-12-23T22:01:05.313ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "26e7fee5-0271-476b-9dca-4658cecd20eb", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
28872025-12-23T22:01:05.313ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
28882025-12-23T22:01:05.316ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3d70673b-9b72-4d15-82e5-b0bc76f4792e", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
28892025-12-23T22:01:05.316ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
28902025-12-23T22:01:05.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28912025-12-23T22:01:05.317ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
28922025-12-23T22:01:05.317ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
28932025-12-23T22:01:05.317ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9ee6a31b-f51d-4253-ad89-cb4ebb863fbc", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
28942025-12-23T22:01:05.317ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28952025-12-23T22:01:05.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28962025-12-23T22:01:05.318ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "75e6f551-91c7-49d8-a3a2-0cad630903ca", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
28972025-12-23T22:01:05.318ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
28982025-12-23T22:01:05.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
28992025-12-23T22:01:05.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29002025-12-23T22:01:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29012025-12-23T22:01:05.321ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
29022025-12-23T22:01:05.321ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 701.480541ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
29032025-12-23T22:01:05.321ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29042025-12-23T22:01:05.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29052025-12-23T22:01:05.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29062025-12-23T22:01:05.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29072025-12-23T22:01:05.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29082025-12-23T22:01:05.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29092025-12-23T22:01:05.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29102025-12-23T22:01:05.327ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29112025-12-23T22:01:05.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29122025-12-23T22:01:05.328ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29132025-12-23T22:01:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
29142025-12-23T22:01:05.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29152025-12-23T22:01:05.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29162025-12-23T22:01:05.329ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
29172025-12-23T22:01:05.329ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29182025-12-23T22:01:05.330ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "97ac72c6-6158-4c04-9bfc-7ffc66587128", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
29192025-12-23T22:01:05.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29202025-12-23T22:01:05.331ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29212025-12-23T22:01:05.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29222025-12-23T22:01:05.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29232025-12-23T22:01:05.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29242025-12-23T22:01:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29252025-12-23T22:01:05.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29262025-12-23T22:01:05.332ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 575.382733ms iteration = 34 status = {}
29272025-12-23T22:01:05.332ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Dependency
29282025-12-23T22:01:05.332ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
29292025-12-23T22:01:05.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29302025-12-23T22:01:05.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29312025-12-23T22:01:05.333ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29322025-12-23T22:01:05.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29332025-12-23T22:01:05.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29342025-12-23T22:01:05.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29352025-12-23T22:01:05.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29362025-12-23T22:01:05.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29372025-12-23T22:01:05.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29382025-12-23T22:01:05.337ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29392025-12-23T22:01:05.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29402025-12-23T22:01:05.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29412025-12-23T22:01:05.337ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1563, tv_nsec: 437140284 } })
29422025-12-23T22:01:05.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29432025-12-23T22:01:05.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29442025-12-23T22:01:05.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
29452025-12-23T22:01:05.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29462025-12-23T22:01:05.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29472025-12-23T22:01:05.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29482025-12-23T22:01:05.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29492025-12-23T22:01:05.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
29502025-12-23T22:01:05.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29512025-12-23T22:01:05.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29522025-12-23T22:01:05.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29532025-12-23T22:01:05.342ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29542025-12-23T22:01:05.342ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29552025-12-23T22:01:05.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29562025-12-23T22:01:05.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29572025-12-23T22:01:05.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29582025-12-23T22:01:05.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29592025-12-23T22:01:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
29602025-12-23T22:01:05.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29612025-12-23T22:01:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29622025-12-23T22:01:05.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29632025-12-23T22:01:05.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29642025-12-23T22:01:05.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29652025-12-23T22:01:05.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29662025-12-23T22:01:05.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29672025-12-23T22:01:05.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29682025-12-23T22:01:05.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29692025-12-23T22:01:05.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29702025-12-23T22:01:05.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29712025-12-23T22:01:05.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29722025-12-23T22:01:05.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29732025-12-23T22:01:05.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
29742025-12-23T22:01:05.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29752025-12-23T22:01:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29762025-12-23T22:01:05.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29772025-12-23T22:01:05.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29782025-12-23T22:01:05.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29792025-12-23T22:01:05.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29802025-12-23T22:01:05.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29812025-12-23T22:01:05.373ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
29822025-12-23T22:01:05.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29832025-12-23T22:01:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29842025-12-23T22:01:05.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29852025-12-23T22:01:05.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29862025-12-23T22:01:05.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29872025-12-23T22:01:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29882025-12-23T22:01:05.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29892025-12-23T22:01:05.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29902025-12-23T22:01:05.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29912025-12-23T22:01:05.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29922025-12-23T22:01:05.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29932025-12-23T22:01:05.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29942025-12-23T22:01:05.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29952025-12-23T22:01:05.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29962025-12-23T22:01:05.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29972025-12-23T22:01:05.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29982025-12-23T22:01:05.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29992025-12-23T22:01:05.397ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
30002025-12-23T22:01:05.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30012025-12-23T22:01:05.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30022025-12-23T22:01:05.398ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
30032025-12-23T22:01:05.398ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
30042025-12-23T22:01:05.398ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9997275f-f72e-4263-b2ef-646f8f341af7 parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
30052025-12-23T22:01:05.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30062025-12-23T22:01:05.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30072025-12-23T22:01:05.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30082025-12-23T22:01:05.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30092025-12-23T22:01:05.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30102025-12-23T22:01:05.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30112025-12-23T22:01:05.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30122025-12-23T22:01:05.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30132025-12-23T22:01:05.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30142025-12-23T22:01:05.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30152025-12-23T22:01:05.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30162025-12-23T22:01:05.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
30172025-12-23T22:01:05.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30182025-12-23T22:01:05.409ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
30192025-12-23T22:01:05.409ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
30202025-12-23T22:01:05.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30212025-12-23T22:01:05.409ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 769.350117ms iteration = 28 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
30222025-12-23T22:01:05.420ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
30232025-12-23T22:01:05.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30242025-12-23T22:01:05.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30252025-12-23T22:01:05.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30262025-12-23T22:01:05.425ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
30272025-12-23T22:01:05.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30282025-12-23T22:01:05.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30292025-12-23T22:01:05.428ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
30302025-12-23T22:01:05.428ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
30312025-12-23T22:01:05.428ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
30322025-12-23T22:01:05.431ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30332025-12-23T22:01:05.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30342025-12-23T22:01:05.434ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
30352025-12-23T22:01:05.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30362025-12-23T22:01:05.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30372025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
30382025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
30392025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
30402025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
30412025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
30422025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
30432025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
30442025-12-23T22:01:05.447ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
30452025-12-23T22:01:05.448ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
30462025-12-23T22:01:05.448ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
30472025-12-23T22:01:05.448ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
30482025-12-23T22:01:05.448ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30492025-12-23T22:01:05.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30502025-12-23T22:01:05.452ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
30512025-12-23T22:01:05.452ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
30522025-12-23T22:01:05.500ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30532025-12-23T22:01:05.504ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30542025-12-23T22:01:05.505ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30552025-12-23T22:01:05.507ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4caa0796-afa7-43ae-a8c5-bfdd86d5b66e", "content-length": "372", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
30562025-12-23T22:01:05.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30572025-12-23T22:01:05.507ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e85cdee3-bc3c-4e97-9c69-cad2b5eea1bd", "content-length": "691", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
30582025-12-23T22:01:05.507ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
30592025-12-23T22:01:05.507ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
30602025-12-23T22:01:05.507ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
30612025-12-23T22:01:05.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30622025-12-23T22:01:05.509ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 751.609926ms iteration = 23 status = {}
30632025-12-23T22:01:05.524ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
30642025-12-23T22:01:05.524ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
30652025-12-23T22:01:05.525ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "dd6d202a-2c23-4b3f-95ce-b5a9e94d8739", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
30662025-12-23T22:01:05.525ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30672025-12-23T22:01:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30682025-12-23T22:01:05.528ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30692025-12-23T22:01:05.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30702025-12-23T22:01:05.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30712025-12-23T22:01:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30722025-12-23T22:01:05.535ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
30732025-12-23T22:01:05.535ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30742025-12-23T22:01:05.542ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5eb2e50e-7da0-4bb1-930e-59f91a653012", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
30752025-12-23T22:01:05.544ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 211.532991ms iteration = 35 status = {}
30762025-12-23T22:01:05.596ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), time_created: 2025-12-23T21:49:57.392343Z, time_modified: 2025-12-23T21:49:57.392343Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
30772025-12-23T22:01:05.596ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
30782025-12-23T22:01:05.602ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
30792025-12-23T22:01:05.602ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
30802025-12-23T22:01:05.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30812025-12-23T22:01:05.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30822025-12-23T22:01:05.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30832025-12-23T22:01:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30842025-12-23T22:01:05.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30852025-12-23T22:01:05.665ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
30862025-12-23T22:01:05.665ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:39113
30872025-12-23T22:01:05.665ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:39113 req_id = 20bbd3a6-91b5-4920-8ab3-f33e1ab2659c uri = /metrics/producers version = 1.0.0
30882025-12-23T22:01:05.665ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:39113 req_id = 20bbd3a6-91b5-4920-8ab3-f33e1ab2659c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
30892025-12-23T22:01:05.665ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:39113 req_id = 20bbd3a6-91b5-4920-8ab3-f33e1ab2659c resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
30902025-12-23T22:01:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
30912025-12-23T22:01:05.667ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
30922025-12-23T22:01:05.668ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ba9e7b18-cc8d-49e6-ab7e-e8a5f5d4a318", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
30932025-12-23T22:01:05.670ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
30942025-12-23T22:01:05.670ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = Dns step id = Ensure
30952025-12-23T22:01:05.670ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
30962025-12-23T22:01:05.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30972025-12-23T22:01:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30982025-12-23T22:01:05.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30992025-12-23T22:01:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31002025-12-23T22:01:05.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31012025-12-23T22:01:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31022025-12-23T22:01:05.678ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
31032025-12-23T22:01:05.678ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
31042025-12-23T22:01:05.678ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
31052025-12-23T22:01:05.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31062025-12-23T22:01:05.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31072025-12-23T22:01:05.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31082025-12-23T22:01:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31092025-12-23T22:01:05.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31102025-12-23T22:01:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31112025-12-23T22:01:05.704ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "82d9f7c8-296f-4b97-b5a0-40d6333d4a02", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
31122025-12-23T22:01:05.704ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
31132025-12-23T22:01:05.704ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31142025-12-23T22:01:05.710ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0e3dd0d-9fa7-40e6-b491-37cbd2751d9c", "content-length": "682", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
31152025-12-23T22:01:05.710ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31162025-12-23T22:01:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31172025-12-23T22:01:05.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31182025-12-23T22:01:05.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31192025-12-23T22:01:05.716ZINFOnexus: assigned collector to new producer collector_id = f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f file = nexus/src/app/oximeter.rs:76 producer_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
31202025-12-23T22:01:05.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 50805 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:39113 req_id = 20bbd3a6-91b5-4920-8ab3-f33e1ab2659c response_code = 201 uri = /metrics/producers
31212025-12-23T22:01:05.716ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "20bbd3a6-91b5-4920-8ab3-f33e1ab2659c", "content-length": "41", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
31222025-12-23T22:01:05.717ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
31232025-12-23T22:01:05.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31242025-12-23T22:01:05.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31252025-12-23T22:01:05.721ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31262025-12-23T22:01:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31272025-12-23T22:01:05.727ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
31282025-12-23T22:01:05.727ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
31292025-12-23T22:01:05.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31302025-12-23T22:01:05.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31312025-12-23T22:01:05.728ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 970.125545ms iteration = 23 status = {}
31322025-12-23T22:01:05.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31332025-12-23T22:01:05.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31342025-12-23T22:01:05.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31352025-12-23T22:01:05.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31362025-12-23T22:01:05.735ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
31372025-12-23T22:01:05.735ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
31382025-12-23T22:01:05.735ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
31392025-12-23T22:01:05.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31402025-12-23T22:01:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31412025-12-23T22:01:05.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31422025-12-23T22:01:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31432025-12-23T22:01:05.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31442025-12-23T22:01:05.741ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31452025-12-23T22:01:05.742ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
31462025-12-23T22:01:05.742ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
31472025-12-23T22:01:05.742ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
31482025-12-23T22:01:05.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31492025-12-23T22:01:05.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31502025-12-23T22:01:05.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31512025-12-23T22:01:05.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31522025-12-23T22:01:05.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31532025-12-23T22:01:05.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31542025-12-23T22:01:05.749ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
31552025-12-23T22:01:05.749ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
31562025-12-23T22:01:05.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31572025-12-23T22:01:05.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31582025-12-23T22:01:05.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31592025-12-23T22:01:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31602025-12-23T22:01:05.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31612025-12-23T22:01:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31622025-12-23T22:01:05.756ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
31632025-12-23T22:01:05.756ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
31642025-12-23T22:01:05.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31652025-12-23T22:01:05.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31662025-12-23T22:01:05.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31672025-12-23T22:01:05.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
31682025-12-23T22:01:05.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31692025-12-23T22:01:05.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31702025-12-23T22:01:05.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31712025-12-23T22:01:05.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31722025-12-23T22:01:05.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31732025-12-23T22:01:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31742025-12-23T22:01:05.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31752025-12-23T22:01:05.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
31762025-12-23T22:01:05.769ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
31772025-12-23T22:01:05.770ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
31782025-12-23T22:01:05.770ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = OmicronZones step id = Cleanup
31792025-12-23T22:01:05.770ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = OmicronZones step id = Cleanup
31802025-12-23T22:01:05.770ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = PhysicalDisks step id = Cleanup
31812025-12-23T22:01:05.770ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = Clickhouse step id = Ensure
31822025-12-23T22:01:05.770ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = Clickhouse step id = Ensure
31832025-12-23T22:01:05.801ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
31842025-12-23T22:01:05.806ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "73cdbac2-afe5-413a-aedb-9536f0d2cc2c", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
31852025-12-23T22:01:05.808ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = SupportBundles step id = Cleanup
31862025-12-23T22:01:05.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31872025-12-23T22:01:05.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31882025-12-23T22:01:05.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31892025-12-23T22:01:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
31902025-12-23T22:01:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31912025-12-23T22:01:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31922025-12-23T22:01:05.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31932025-12-23T22:01:05.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31942025-12-23T22:01:05.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31952025-12-23T22:01:05.819ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
31962025-12-23T22:01:05.823ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = OmicronZones step id = Cleanup
31972025-12-23T22:01:05.823ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
31982025-12-23T22:01:05.823ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
31992025-12-23T22:01:05.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32002025-12-23T22:01:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32012025-12-23T22:01:05.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32022025-12-23T22:01:05.826ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
32032025-12-23T22:01:05.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32042025-12-23T22:01:05.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32052025-12-23T22:01:05.870ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
32062025-12-23T22:01:05.870ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(02039deb-aee8-405c-80d5-e584bd199fe1 (blueprint), false)
32072025-12-23T22:01:05.870ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = Cockroach step id = Ensure
32082025-12-23T22:01:05.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32092025-12-23T22:01:05.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32102025-12-23T22:01:05.886ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
32112025-12-23T22:01:05.886ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 3047a85f-a0c0-41ad-af77-caf28e7543b1 step component = MgsUpdates step id = Ensure
32122025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 35 reason = Dependency
32132025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Dependency
32142025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32152025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32162025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.267381475s iteration = 13 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2760,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":70981164,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":70981114,"secs":0}},"event_index":1,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":70991754,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":22876516,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":22876466,"secs":0}},"event_index":2,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":93886130,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":46638156,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":46638116,"secs":0}},"event_index":3,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":140543627,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":374305086,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":374305046,"secs":0}},"event_index":4,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":514867484,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":536208526,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":536208496,"secs":0}},"event_index":5,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":51087840,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":99046546,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":99046506,"secs":0}},"event_index":6,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150153246,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4271,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4241,"secs":0}},"event_index":7,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150168827,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2390,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2360,"secs":0}},"event_index":8,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150173327,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3420,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3380,"secs":0}},"event_index":9,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150179017,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1800,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1760,"secs":0}},"event_index":10,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150182217,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38095006,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38094966,"secs":0}},"event_index":11,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188278473,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14870370,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14870320,"secs":0}},"event_index":12,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":203163383,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47564613,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":47564573,"secs":0}},"event_index":13,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":250740936,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16312371,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16312331,"secs":0}},"event_index":14,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":267067846,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4700,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4670,"secs":0}},"event_index":15,"execution_id":"1da7f60f-15d6-4f7b-ad72-19ac142d7dfe","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":267079887,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"02039deb-aee8-405c-80d5-e584bd199fe1"}
32172025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32182025-12-23T22:01:05.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
32192025-12-23T22:01:05.889ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32202025-12-23T22:01:05.889ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32212025-12-23T22:01:05.892ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
32222025-12-23T22:01:05.892ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32232025-12-23T22:01:05.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32242025-12-23T22:01:05.892ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32252025-12-23T22:01:05.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
32262025-12-23T22:01:05.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32272025-12-23T22:01:05.932ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32282025-12-23T22:01:05.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32292025-12-23T22:01:05.935ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32302025-12-23T22:01:05.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32312025-12-23T22:01:05.938ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32322025-12-23T22:01:05.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32332025-12-23T22:01:05.940ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
32342025-12-23T22:01:05.940ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 53.371817ms iteration = 35 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
32352025-12-23T22:01:05.957ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(c7d335e4-0197-415a-bfdb-e9e304196cc2, 0), (44ce8ae2-e1ec-40b1-b571-85365dc3d6dd, 0), (f92b2309-ea11-4dbe-892d-2a47b29ba690, 0)]
32362025-12-23T22:01:05.957ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1564, tv_nsec: 57503166 } })
32372025-12-23T22:01:05.990ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("01b204c0-22af-4353-ad94-4c2c0d1eb01b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70f89219-475c-48c6-a8c8-388723cccdcf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("7b633ce6-0ad9-42aa-8164-bc5c5f301b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("eaafa87a-91ad-4109-aa73-7065c531ed97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1564, tv_nsec: 90060545 } })
32382025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
32392025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
32402025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
32412025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d64aad57-0da7-4315-9cbb-c47c768832d4", "content-length": "476", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32422025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
32432025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "29080666-4bb7-4be9-b471-4e18f8acde92", "content-length": "734", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32442025-12-23T22:01:05.991ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
32452025-12-23T22:01:05.992ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3db77741-f386-45d3-aaea-a079361970d1", "content-length": "111", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32462025-12-23T22:01:05.992ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
32472025-12-23T22:01:05.992ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbe724ea-f2ce-4238-9481-fa9fd228696d", "content-length": "111", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32482025-12-23T22:01:05.992ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
32492025-12-23T22:01:05.993ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41fcc8d6-7c5e-45cb-b3ec-fcc3aa1583ed", "content-length": "163", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32502025-12-23T22:01:05.993ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
32512025-12-23T22:01:05.993ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20508fbc-cda0-40cf-af01-934ddee12b43", "content-length": "163", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32522025-12-23T22:01:05.993ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
32532025-12-23T22:01:05.993ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5470ef38-17a9-4c67-b419-605bce7ba986", "content-length": "166", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32542025-12-23T22:01:05.993ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
32552025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb53b20e-3868-43c4-af3b-42016c1e27c8", "content-length": "166", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32562025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
32572025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "805765fb-edd7-4bf6-864b-f3b763c5c01b", "content-length": "702", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32582025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
32592025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30c6f7f2-bd38-41d2-8b15-eededec29ba2", "content-length": "727", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32602025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
32612025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc967626-fe5a-454a-9e25-aeb1d3800eee", "content-length": "729", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32622025-12-23T22:01:05.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
32632025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f48c3b5-3970-47b5-8241-41f595ae920a", "content-length": "728", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32642025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
32652025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6bf10f1-13c1-4ea9-8d4a-f756340cf6ad", "content-length": "722", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32662025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
32672025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bb28d7f-348c-4d15-9154-391882a83cfe", "content-length": "10", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32682025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
32692025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04c101b1-ece0-41bb-817f-8ae2c428fee5", "content-length": "144", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32702025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
32712025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "141bcbc8-fe03-4976-917d-f2c924a88822", "content-length": "147", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32722025-12-23T22:01:05.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
32732025-12-23T22:01:05.996ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa8100a3-1825-4f9e-a9c5-dddb96a63fb9", "content-length": "109", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32742025-12-23T22:01:05.996ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
32752025-12-23T22:01:05.996ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a1c627e-1ac0-40fe-96a9-351918a66f64", "content-length": "109", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32762025-12-23T22:01:05.996ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
32772025-12-23T22:01:05.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51a1f982-3516-427b-9477-75cdc1827509", "content-length": "163", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32782025-12-23T22:01:05.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
32792025-12-23T22:01:05.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00a1fe50-d18d-467d-b69c-bb27967ffbe2", "content-length": "163", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32802025-12-23T22:01:05.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
32812025-12-23T22:01:05.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1ea693f-c2bc-4004-9d96-b96c7e550ae8", "content-length": "166", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32822025-12-23T22:01:05.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
32832025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8269a651-ac58-4f81-ad4d-023d182be452", "content-length": "166", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32842025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
32852025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14ac449d-fbb2-42cc-b624-8f71ced57a9f", "content-length": "702", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32862025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
32872025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bcd5421-edbc-471a-9275-4258ab3a0308", "content-length": "727", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32882025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
32892025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc04b1e9-5d4f-4892-bbd1-88a815ecf92a", "content-length": "729", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32902025-12-23T22:01:05.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
32912025-12-23T22:01:05.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe8ee863-f40b-44da-a68b-e6e62c4f3033", "content-length": "728", "date": "Tue, 23 Dec 2025 22:01:05 GMT"} })
32922025-12-23T22:01:05.999ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32932025-12-23T22:01:05.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
32942025-12-23T22:01:06.001ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32952025-12-23T22:01:06.003ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
32962025-12-23T22:01:06.003ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32972025-12-23T22:01:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
32982025-12-23T22:01:06.005ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32992025-12-23T22:01:06.007ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
33002025-12-23T22:01:06.007ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33012025-12-23T22:01:06.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
33022025-12-23T22:01:06.044ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
33032025-12-23T22:01:06.044ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
33042025-12-23T22:01:06.137ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6fa9a7e-22a6-4fa4-a3c6-6ac6fd6a78c6", "content-length": "95745", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33052025-12-23T22:01:06.140ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
33062025-12-23T22:01:06.140ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
33072025-12-23T22:01:06.140ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
33082025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
33092025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
33102025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
33112025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
33122025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
33132025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f33ac50-b1b3-4773-8cb8-e5782fb1eb43", "content-length": "64", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33142025-12-23T22:01:06.140ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1adfef59-fa3a-4e90-a5e9-edb68975506a", "content-length": "64", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33152025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cb7c82f-c307-460b-978c-89a61ce23d3f", "content-length": "64", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33162025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
33172025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
33182025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
33192025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c867e9e-61ac-4174-bef5-cfe9ad0d4e35", "content-length": "64", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33202025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
33212025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "16d2dada-dca7-432c-823d-c07cf6e2d402", "content-length": "64", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33222025-12-23T22:01:06.141ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
33232025-12-23T22:01:06.180ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1988e67e-c11b-499d-a1ea-809b070cc135", "content-length": "416423", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33242025-12-23T22:01:06.181ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bec7dcb0-1234-4c3a-b244-dcdefa8ba3c7", "content-length": "414123", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33252025-12-23T22:01:06.181ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8df8b7a1-19f3-4567-a419-acdfcc260622", "content-length": "409535", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33262025-12-23T22:01:06.184ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae5c056e-7fcc-4e2d-ab43-58f70ba96073", "content-length": "410450", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33272025-12-23T22:01:06.185ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1755667-66c7-4ef8-b1e6-42762cb18949", "content-length": "428872", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33282025-12-23T22:01:06.188ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
33292025-12-23T22:01:06.188ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
33302025-12-23T22:01:06.188ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
33312025-12-23T22:01:06.188ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
33322025-12-23T22:01:06.188ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
33332025-12-23T22:01:06.218ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c (omicron_zone)
33342025-12-23T22:01:06.218ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
33352025-12-23T22:01:06.222ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd6d5c9a-424c-4590-a802-b4e2254b1915", "content-length": "127", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33362025-12-23T22:01:06.223ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
33372025-12-23T22:01:06.317ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
33382025-12-23T22:01:06.317ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
33392025-12-23T22:01:06.317ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "791ca2cd-e2da-42b5-a8a1-7bd446714221", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33402025-12-23T22:01:06.317ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
33412025-12-23T22:01:06.319ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
33422025-12-23T22:01:06.319ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
33432025-12-23T22:01:06.320ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb505cac-db73-4e53-aff3-4260b3737489", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33442025-12-23T22:01:06.320ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
33452025-12-23T22:01:06.321ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
33462025-12-23T22:01:06.321ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
33472025-12-23T22:01:06.322ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d10e0ed-d669-42d0-b02e-56e085741ebf", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:01:06 GMT"} })
33482025-12-23T22:01:06.322ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
33492025-12-23T22:01:06.323ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
33502025-12-23T22:01:06.323ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
33512025-12-23T22:01:06.325ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33522025-12-23T22:01:06.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
33532025-12-23T22:01:06.327ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33542025-12-23T22:01:06.330ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
33552025-12-23T22:01:06.330ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33562025-12-23T22:01:06.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
33572025-12-23T22:01:06.516ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = c52dbd67-418f-4c46-9f2c-a5d9fa4bf644 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
33582025-12-23T22:01:06.516ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = c52dbd67-418f-4c46-9f2c-a5d9fa4bf644 time_started = 2025-12-23 22:01:05.991006 UTC
33592025-12-23T22:01:06.516ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 41 reason = Dependency
33602025-12-23T22:01:06.516ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
33612025-12-23T22:01:06.517ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 24.931\u{b5}s iteration = 41 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
33622025-12-23T22:01:06.517ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 60 reason = Dependency
33632025-12-23T22:01:06.517ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
33642025-12-23T22:01:06.517ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 629.997923ms iteration = 15 status = {"collection_id":"c52dbd67-418f-4c46-9f2c-a5d9fa4bf644","time_done":"2025-12-23 22:01:06.323561 UTC","time_started":"2025-12-23 22:01:05.991006 UTC"}
33652025-12-23T22:01:06.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
33662025-12-23T22:01:06.519ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33672025-12-23T22:01:06.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
33682025-12-23T22:01:06.525ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
33692025-12-23T22:01:06.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
33702025-12-23T22:01:06.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
33712025-12-23T22:01:06.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
33722025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 162.024303ms iteration = 60 status = {"Loaded":{"collection_id":"7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e","time_loaded":"2025-12-23T22:01:06.516993385Z","time_started":"2025-12-23T22:01:06.107605Z"}}
33732025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 18 reason = Dependency
33742025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 29 reason = Dependency
33752025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 42 reason = Dependency
33762025-12-23T22:01:06.679ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
33772025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33782025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
33792025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 38 reason = Dependency
33802025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.821\u{b5}s iteration = 42 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
33812025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33822025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33832025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33842025-12-23T22:01:06.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
33852025-12-23T22:01:06.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33862025-12-23T22:01:06.681ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33872025-12-23T22:01:06.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33882025-12-23T22:01:06.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33892025-12-23T22:01:06.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33902025-12-23T22:01:06.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33912025-12-23T22:01:06.685ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
33922025-12-23T22:01:06.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33932025-12-23T22:01:06.685ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
33942025-12-23T22:01:06.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
33952025-12-23T22:01:06.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33962025-12-23T22:01:06.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
33972025-12-23T22:01:06.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33982025-12-23T22:01:06.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33992025-12-23T22:01:06.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34002025-12-23T22:01:06.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34012025-12-23T22:01:06.688ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34022025-12-23T22:01:06.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
34032025-12-23T22:01:06.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34042025-12-23T22:01:06.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34052025-12-23T22:01:06.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
34062025-12-23T22:01:06.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34072025-12-23T22:01:06.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34082025-12-23T22:01:06.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34092025-12-23T22:01:06.692ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34102025-12-23T22:01:06.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34112025-12-23T22:01:06.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34122025-12-23T22:01:06.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34132025-12-23T22:01:06.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34142025-12-23T22:01:06.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
34152025-12-23T22:01:06.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34162025-12-23T22:01:06.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34172025-12-23T22:01:06.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34182025-12-23T22:01:06.700ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
34192025-12-23T22:01:06.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34202025-12-23T22:01:06.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34212025-12-23T22:01:06.703ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
34222025-12-23T22:01:06.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34232025-12-23T22:01:06.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34242025-12-23T22:01:06.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34252025-12-23T22:01:06.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
34262025-12-23T22:01:06.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34272025-12-23T22:01:06.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34282025-12-23T22:01:06.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34292025-12-23T22:01:06.711ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
34302025-12-23T22:01:06.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
34312025-12-23T22:01:06.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
34322025-12-23T22:01:06.712ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
34332025-12-23T22:01:06.712ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.831405ms iteration = 18 status = {"blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","inventory_collection_id":"7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
34342025-12-23T22:01:06.739ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 60.408005ms iteration = 38 status = {"physical_disks_added":0}
34352025-12-23T22:01:06.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34362025-12-23T22:01:06.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34372025-12-23T22:01:06.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34382025-12-23T22:01:06.776ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
34392025-12-23T22:01:06.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34402025-12-23T22:01:06.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34412025-12-23T22:01:06.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34422025-12-23T22:01:06.780ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
34432025-12-23T22:01:06.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34442025-12-23T22:01:06.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34452025-12-23T22:01:06.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34462025-12-23T22:01:06.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34472025-12-23T22:01:06.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34482025-12-23T22:01:06.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34492025-12-23T22:01:06.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34502025-12-23T22:01:06.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
34512025-12-23T22:01:06.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34522025-12-23T22:01:06.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34532025-12-23T22:01:06.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34542025-12-23T22:01:06.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34552025-12-23T22:01:06.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34562025-12-23T22:01:06.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
34572025-12-23T22:01:06.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34582025-12-23T22:01:06.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34592025-12-23T22:01:06.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34602025-12-23T22:01:06.837ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
34612025-12-23T22:01:06.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34622025-12-23T22:01:06.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34632025-12-23T22:01:06.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34642025-12-23T22:01:06.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34652025-12-23T22:01:06.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34662025-12-23T22:01:06.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
34672025-12-23T22:01:06.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34682025-12-23T22:01:06.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34692025-12-23T22:01:06.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34702025-12-23T22:01:06.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34712025-12-23T22:01:06.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34722025-12-23T22:01:06.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
34732025-12-23T22:01:06.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34742025-12-23T22:01:06.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34752025-12-23T22:01:06.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34762025-12-23T22:01:06.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34772025-12-23T22:01:06.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34782025-12-23T22:01:06.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
34792025-12-23T22:01:06.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34802025-12-23T22:01:06.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34812025-12-23T22:01:06.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34822025-12-23T22:01:06.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34832025-12-23T22:01:06.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34842025-12-23T22:01:06.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34852025-12-23T22:01:06.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34862025-12-23T22:01:06.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
34872025-12-23T22:01:06.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34882025-12-23T22:01:06.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34892025-12-23T22:01:06.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34902025-12-23T22:01:06.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
34912025-12-23T22:01:06.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34922025-12-23T22:01:06.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34932025-12-23T22:01:06.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34942025-12-23T22:01:06.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34952025-12-23T22:01:06.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
34962025-12-23T22:01:06.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
34972025-12-23T22:01:06.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34982025-12-23T22:01:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
34992025-12-23T22:01:06.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35002025-12-23T22:01:06.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35012025-12-23T22:01:06.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35022025-12-23T22:01:06.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
35032025-12-23T22:01:06.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35042025-12-23T22:01:06.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35052025-12-23T22:01:06.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35062025-12-23T22:01:06.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35072025-12-23T22:01:06.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35082025-12-23T22:01:06.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35092025-12-23T22:01:06.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35102025-12-23T22:01:06.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
35112025-12-23T22:01:06.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35122025-12-23T22:01:06.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35132025-12-23T22:01:06.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35142025-12-23T22:01:06.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35152025-12-23T22:01:06.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35162025-12-23T22:01:06.917ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
35172025-12-23T22:01:06.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35182025-12-23T22:01:06.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35192025-12-23T22:01:06.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35202025-12-23T22:01:06.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35212025-12-23T22:01:06.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35222025-12-23T22:01:06.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
35232025-12-23T22:01:06.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35242025-12-23T22:01:06.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35252025-12-23T22:01:06.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35262025-12-23T22:01:06.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35272025-12-23T22:01:06.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35282025-12-23T22:01:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
35292025-12-23T22:01:06.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35302025-12-23T22:01:06.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35312025-12-23T22:01:06.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35322025-12-23T22:01:06.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35332025-12-23T22:01:06.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35342025-12-23T22:01:06.941ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
35352025-12-23T22:01:06.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35362025-12-23T22:01:06.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35372025-12-23T22:01:06.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35382025-12-23T22:01:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35392025-12-23T22:01:06.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35402025-12-23T22:01:06.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
35412025-12-23T22:01:06.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35422025-12-23T22:01:06.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35432025-12-23T22:01:06.983ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
35442025-12-23T22:01:06.983ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
35452025-12-23T22:01:06.983ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 38108585-6375-4291-9db4-f5c5b388542e parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
35462025-12-23T22:01:06.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35472025-12-23T22:01:06.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35482025-12-23T22:01:06.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35492025-12-23T22:01:06.988ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
35502025-12-23T22:01:06.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35512025-12-23T22:01:06.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35522025-12-23T22:01:06.991ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
35532025-12-23T22:01:06.991ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
35542025-12-23T22:01:06.991ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 312.10038ms iteration = 29 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
35552025-12-23T22:01:09.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 134 reason = Timeout
35562025-12-23T22:01:09.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35572025-12-23T22:01:09.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35582025-12-23T22:01:09.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35592025-12-23T22:01:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
35602025-12-23T22:01:09.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35612025-12-23T22:01:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35622025-12-23T22:01:09.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
35632025-12-23T22:01:09.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.452271ms iteration = 134 status = {"config_updated":false}
35642025-12-23T22:01:11.823ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 439 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 8902be6f-beb2-4957-a959-82dcc5af778c response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
35652025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
35662025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
35672025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35682025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35692025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35702025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35712025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 135 reason = Timeout
35722025-12-23T22:01:14.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35732025-12-23T22:01:14.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35742025-12-23T22:01:14.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35752025-12-23T22:01:14.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35762025-12-23T22:01:14.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35772025-12-23T22:01:14.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35782025-12-23T22:01:14.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.028693ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35792025-12-23T22:01:14.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35802025-12-23T22:01:14.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35812025-12-23T22:01:14.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35822025-12-23T22:01:14.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
35832025-12-23T22:01:14.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35842025-12-23T22:01:14.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35852025-12-23T22:01:14.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35862025-12-23T22:01:14.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
35872025-12-23T22:01:14.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.600215ms iteration = 135 status = {"config_updated":false}
35882025-12-23T22:01:14.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35892025-12-23T22:01:14.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35902025-12-23T22:01:14.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
35912025-12-23T22:01:14.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35922025-12-23T22:01:14.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35932025-12-23T22:01:14.785ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
35942025-12-23T22:01:14.785ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.176372ms iteration = 68 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
35952025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 61 reason = Timeout
35962025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 46 reason = Timeout
35972025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
35982025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 46 reason = Timeout
35992025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
36002025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
36012025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 136 reason = Timeout
36022025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36032025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
36042025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36052025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
36062025-12-23T22:01:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
36072025-12-23T22:01:19.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
36082025-12-23T22:01:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
36092025-12-23T22:01:19.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36102025-12-23T22:01:19.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36112025-12-23T22:01:19.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36122025-12-23T22:01:19.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36132025-12-23T22:01:19.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
36142025-12-23T22:01:19.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
36152025-12-23T22:01:19.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
36162025-12-23T22:01:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
36172025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
36182025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
36192025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
36202025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36212025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
36222025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36232025-12-23T22:01:19.763ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
36242025-12-23T22:01:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
36252025-12-23T22:01:19.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.836751ms iteration = 46 status = "NoSitrep"
36262025-12-23T22:01:19.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
36272025-12-23T22:01:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
36282025-12-23T22:01:19.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
36292025-12-23T22:01:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.86707ms iteration = 136 status = {"config_updated":false}
36302025-12-23T22:01:19.765ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e original_time_started = 2025-12-23 22:01:06.107605 UTC
36312025-12-23T22:01:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.020875ms iteration = 61 status = {"Loaded":{"collection_id":"7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e","time_loaded":"2025-12-23T22:01:19.756474457Z","time_started":"2025-12-23T22:01:06.107605Z"}}
36322025-12-23T22:01:19.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36332025-12-23T22:01:19.767ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
36342025-12-23T22:01:19.768ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
36352025-12-23T22:01:19.768ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
36362025-12-23T22:01:19.768ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.376919ms iteration = 46 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
36372025-12-23T22:01:21.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 469 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 41e01657-f418-45d1-aabb-77d8df5b99c3 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
36382025-12-23T22:01:24.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
36392025-12-23T22:01:24.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36402025-12-23T22:01:24.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
36412025-12-23T22:01:24.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36422025-12-23T22:01:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36432025-12-23T22:01:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
36442025-12-23T22:01:24.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 137 reason = Timeout
36452025-12-23T22:01:24.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36462025-12-23T22:01:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36472025-12-23T22:01:24.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36482025-12-23T22:01:24.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36492025-12-23T22:01:24.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36502025-12-23T22:01:24.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36512025-12-23T22:01:24.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36522025-12-23T22:01:24.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36532025-12-23T22:01:24.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
36542025-12-23T22:01:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
36552025-12-23T22:01:24.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36562025-12-23T22:01:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36572025-12-23T22:01:24.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36582025-12-23T22:01:24.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
36592025-12-23T22:01:24.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36602025-12-23T22:01:24.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
36612025-12-23T22:01:24.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.713946ms iteration = 137 status = {"config_updated":false}
36622025-12-23T22:01:24.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
36632025-12-23T22:01:24.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36642025-12-23T22:01:24.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
36652025-12-23T22:01:24.783ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
36662025-12-23T22:01:24.783ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.836736ms iteration = 69 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
36672025-12-23T22:01:24.800ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 44.65ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36682025-12-23T22:01:26.816ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:44873
36692025-12-23T22:01:26.817ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 711d3ce8-7085-478b-94b2-3bd543bafd4a uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending version = 1.0.0
36702025-12-23T22:01:26.817ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 711d3ce8-7085-478b-94b2-3bd543bafd4a roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
36712025-12-23T22:01:26.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 711d3ce8-7085-478b-94b2-3bd543bafd4a resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
36722025-12-23T22:01:26.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2948 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 711d3ce8-7085-478b-94b2-3bd543bafd4a response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending
36732025-12-23T22:01:26.820ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 392b372e-9c04-42f0-ac46-ac3f3987ea5a uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D version = 1.0.0
36742025-12-23T22:01:26.820ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 392b372e-9c04-42f0-ac46-ac3f3987ea5a roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
36752025-12-23T22:01:26.820ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 392b372e-9c04-42f0-ac46-ac3f3987ea5a resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
36762025-12-23T22:01:26.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2096 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44873 req_id = 392b372e-9c04-42f0-ac46-ac3f3987ea5a response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D
36772025-12-23T22:01:29.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 138 reason = Timeout
36782025-12-23T22:01:29.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36792025-12-23T22:01:29.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36802025-12-23T22:01:29.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36812025-12-23T22:01:29.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
36822025-12-23T22:01:29.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36832025-12-23T22:01:29.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36842025-12-23T22:01:29.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
36852025-12-23T22:01:29.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.679109ms iteration = 138 status = {"config_updated":false}
36862025-12-23T22:01:31.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 455 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 5a19701f-22e5-4142-a261-ff7ff11112de response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
36872025-12-23T22:01:34.620ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
36882025-12-23T22:01:34.620ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
36892025-12-23T22:01:34.620ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
36902025-12-23T22:01:34.620ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
36912025-12-23T22:01:34.620ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36922025-12-23T22:01:34.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
36932025-12-23T22:01:34.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
36942025-12-23T22:01:34.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 723924662 } })
36952025-12-23T22:01:34.624ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 724108737 } })
36962025-12-23T22:01:34.624ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
36972025-12-23T22:01:34.625ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 724914590 } })
36982025-12-23T22:01:34.625ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.800475ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36992025-12-23T22:01:34.685ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
37002025-12-23T22:01:34.686ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 65.905229ms iteration = 24 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
37012025-12-23T22:01:34.691ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
37022025-12-23T22:01:34.691ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
37032025-12-23T22:01:34.692ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "906e26c1-0b5b-4db0-93c5-019cf7ac5de8", "content-length": "26", "date": "Tue, 23 Dec 2025 22:01:34 GMT"} })
37042025-12-23T22:01:34.692ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
37052025-12-23T22:01:34.693ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
37062025-12-23T22:01:34.694ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 794193235 } })
37072025-12-23T22:01:34.754ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
37082025-12-23T22:01:34.754ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
37092025-12-23T22:01:34.754ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
37102025-12-23T22:01:34.754ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
37112025-12-23T22:01:34.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37122025-12-23T22:01:34.754ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37132025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37142025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
37152025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
37162025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
37172025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
37182025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37192025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37202025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 134.551608ms iteration = 24 status = {}
37212025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37222025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37232025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
37242025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
37252025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37262025-12-23T22:01:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37272025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 39 reason = Timeout
37282025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 36 reason = Timeout
37292025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
37302025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37312025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37322025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
37332025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37342025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
37352025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
37362025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 47 reason = Timeout
37372025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37382025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
37392025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
37402025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37412025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 36 reason = Timeout
37422025-12-23T22:01:34.756ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
37432025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
37442025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37452025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37462025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 47 reason = Timeout
37472025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
37482025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 139 reason = Timeout
37492025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 62 reason = Timeout
37502025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37512025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
37522025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
37532025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
37542025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
37552025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.59\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
37562025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
37572025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37582025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
37592025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
37602025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37612025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
37622025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
37632025-12-23T22:01:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
37642025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37652025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37662025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37672025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
37682025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
37692025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37702025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
37712025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37722025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37732025-12-23T22:01:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37742025-12-23T22:01:34.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
37752025-12-23T22:01:34.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37762025-12-23T22:01:34.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.451888ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
37772025-12-23T22:01:34.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
37782025-12-23T22:01:34.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37792025-12-23T22:01:34.758ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37802025-12-23T22:01:34.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37812025-12-23T22:01:34.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37822025-12-23T22:01:34.759ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37832025-12-23T22:01:34.759ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37842025-12-23T22:01:34.759ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37852025-12-23T22:01:34.760ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37862025-12-23T22:01:34.761ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 6.616427ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37872025-12-23T22:01:34.761ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37882025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37892025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37902025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37912025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37922025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
37932025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
37942025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37952025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37962025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37972025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
37982025-12-23T22:01:34.766ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37992025-12-23T22:01:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
38002025-12-23T22:01:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38012025-12-23T22:01:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38022025-12-23T22:01:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38032025-12-23T22:01:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
38042025-12-23T22:01:34.767ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38052025-12-23T22:01:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
38062025-12-23T22:01:34.768ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38072025-12-23T22:01:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38082025-12-23T22:01:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38092025-12-23T22:01:34.768ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38102025-12-23T22:01:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
38112025-12-23T22:01:34.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38122025-12-23T22:01:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38132025-12-23T22:01:34.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 13.143571ms iteration = 47 status = "NoSitrep"
38142025-12-23T22:01:34.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38152025-12-23T22:01:34.769ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38162025-12-23T22:01:34.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38172025-12-23T22:01:34.770ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
38182025-12-23T22:01:34.770ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 13.780969ms iteration = 139 status = {"config_updated":false}
38192025-12-23T22:01:34.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38202025-12-23T22:01:34.771ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 871374083 } })
38212025-12-23T22:01:34.772ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.253618ms iteration = 24 status = 0
38222025-12-23T22:01:34.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38232025-12-23T22:01:34.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38242025-12-23T22:01:34.772ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
38252025-12-23T22:01:34.772ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 872433603 } })
38262025-12-23T22:01:34.773ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
38272025-12-23T22:01:34.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38282025-12-23T22:01:34.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38292025-12-23T22:01:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
38302025-12-23T22:01:34.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38312025-12-23T22:01:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38322025-12-23T22:01:34.777ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38332025-12-23T22:01:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38342025-12-23T22:01:34.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38352025-12-23T22:01:34.816ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
38362025-12-23T22:01:34.816ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 61.348812ms iteration = 70 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
38372025-12-23T22:01:34.818ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38382025-12-23T22:01:34.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38392025-12-23T22:01:34.821ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38402025-12-23T22:01:34.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38412025-12-23T22:01:34.825ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38422025-12-23T22:01:34.826ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
38432025-12-23T22:01:34.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38442025-12-23T22:01:34.827ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38452025-12-23T22:01:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
38462025-12-23T22:01:34.831ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38472025-12-23T22:01:34.831ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
38482025-12-23T22:01:34.831ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38492025-12-23T22:01:34.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
38502025-12-23T22:01:34.832ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.089848ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
38512025-12-23T22:01:34.833ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
38522025-12-23T22:01:34.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
38532025-12-23T22:01:34.833ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38542025-12-23T22:01:34.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38552025-12-23T22:01:34.834ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
38562025-12-23T22:01:34.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
38572025-12-23T22:01:34.836ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38582025-12-23T22:01:34.836ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38592025-12-23T22:01:34.836ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
38602025-12-23T22:01:34.836ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38612025-12-23T22:01:34.837ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5847614-7f04-4147-87bf-0861f857708c", "content-length": "26", "date": "Tue, 23 Dec 2025 22:01:34 GMT"} })
38622025-12-23T22:01:34.837ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38632025-12-23T22:01:34.837ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 81.088929ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
38642025-12-23T22:01:34.838ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
38652025-12-23T22:01:34.838ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
38662025-12-23T22:01:34.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
38672025-12-23T22:01:34.838ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38682025-12-23T22:01:34.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
38692025-12-23T22:01:34.839ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
38702025-12-23T22:01:34.839ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 83.214509ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
38712025-12-23T22:01:34.839ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
38722025-12-23T22:01:34.840ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
38732025-12-23T22:01:34.840ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e original_time_started = 2025-12-23 22:01:06.107605 UTC
38742025-12-23T22:01:34.840ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 83.696943ms iteration = 62 status = {"Loaded":{"collection_id":"7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e","time_loaded":"2025-12-23T22:01:34.756461386Z","time_started":"2025-12-23T22:01:06.107605Z"}}
38752025-12-23T22:01:34.841ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38762025-12-23T22:01:34.841ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
38772025-12-23T22:01:34.841ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38782025-12-23T22:01:34.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
38792025-12-23T22:01:34.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38802025-12-23T22:01:34.841ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38812025-12-23T22:01:34.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38822025-12-23T22:01:34.842ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
38832025-12-23T22:01:34.842ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
38842025-12-23T22:01:34.842ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 86.221784ms iteration = 47 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
38852025-12-23T22:01:34.876ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1592, tv_nsec: 976025007 } })
38862025-12-23T22:01:34.895ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 139.015434ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
38872025-12-23T22:01:34.897ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 142.501023ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
38882025-12-23T22:01:34.903ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
38892025-12-23T22:01:34.903ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 147.388261ms iteration = 36 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
38902025-12-23T22:01:34.906ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38912025-12-23T22:01:34.907ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
38922025-12-23T22:01:34.907ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38932025-12-23T22:01:34.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
38942025-12-23T22:01:34.913ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 157.100275ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
38952025-12-23T22:01:34.920ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1593, tv_nsec: 20446332 } })
38962025-12-23T22:01:34.965ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
38972025-12-23T22:01:34.965ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38982025-12-23T22:01:34.965ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
38992025-12-23T22:01:34.965ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39002025-12-23T22:01:34.965ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 210.488031ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
39012025-12-23T22:01:34.965ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f8ae4e7-a6ca-41e1-b087-92b74ab57682", "content-length": "372", "date": "Tue, 23 Dec 2025 22:01:34 GMT"} })
39022025-12-23T22:01:34.965ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39032025-12-23T22:01:34.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39042025-12-23T22:01:34.975ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39052025-12-23T22:01:34.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39062025-12-23T22:01:34.978ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
39072025-12-23T22:01:34.978ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39082025-12-23T22:01:34.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39092025-12-23T22:01:34.987ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39102025-12-23T22:01:34.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39112025-12-23T22:01:34.992ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 235.804807ms iteration = 39 status = {"physical_disks_added":0}
39122025-12-23T22:01:34.993ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
39132025-12-23T22:01:34.993ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
39142025-12-23T22:01:34.994ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
39152025-12-23T22:01:34.994ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
39162025-12-23T22:01:34.994ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
39172025-12-23T22:01:34.994ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
39182025-12-23T22:01:34.994ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
39192025-12-23T22:01:34.994ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
39202025-12-23T22:01:34.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
39212025-12-23T22:01:34.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
39222025-12-23T22:01:34.995ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
39232025-12-23T22:01:34.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39242025-12-23T22:01:34.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39252025-12-23T22:01:34.998ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
39262025-12-23T22:01:34.998ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
39272025-12-23T22:01:34.999ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39282025-12-23T22:01:34.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39292025-12-23T22:01:35.001ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39302025-12-23T22:01:35.004ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39312025-12-23T22:01:35.004ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39322025-12-23T22:01:35.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39332025-12-23T22:01:35.007ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39342025-12-23T22:01:35.010ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39352025-12-23T22:01:35.010ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39362025-12-23T22:01:35.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39372025-12-23T22:01:35.040ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
39382025-12-23T22:01:35.040ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
39392025-12-23T22:01:35.042ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "dd967330-ada5-4aad-801c-40fb989badb3", "date": "Tue, 23 Dec 2025 22:01:35 GMT"} })
39402025-12-23T22:01:35.042ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39412025-12-23T22:01:35.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39422025-12-23T22:01:35.045ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39432025-12-23T22:01:35.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39442025-12-23T22:01:35.049ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39452025-12-23T22:01:35.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39462025-12-23T22:01:35.056ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
39472025-12-23T22:01:35.056ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39482025-12-23T22:01:35.058ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
39492025-12-23T22:01:35.066ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdd2c815-8117-4d58-ac71-b010cf079287", "content-length": "691", "date": "Tue, 23 Dec 2025 22:01:35 GMT"} })
39502025-12-23T22:01:35.066ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7b478569-d9e5-4a70-8d37-26517c4a87fd", "date": "Tue, 23 Dec 2025 22:01:35 GMT"} })
39512025-12-23T22:01:35.066ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
39522025-12-23T22:01:35.066ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
39532025-12-23T22:01:35.066ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
39542025-12-23T22:01:35.068ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 312.593073ms iteration = 24 status = {}
39552025-12-23T22:01:35.069ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 313.300154ms iteration = 36 status = {}
39562025-12-23T22:01:35.342ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5cba05cd-b258-4fa2-bbe5-25a7a11acfb8", "date": "Tue, 23 Dec 2025 22:01:35 GMT"} })
39572025-12-23T22:01:35.342ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
39582025-12-23T22:01:35.342ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
39592025-12-23T22:01:35.342ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3aaa559a-3c26-4eda-be42-08a37e87eb98", "content-length": "682", "date": "Tue, 23 Dec 2025 22:01:35 GMT"} })
39602025-12-23T22:01:35.342ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39612025-12-23T22:01:35.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39622025-12-23T22:01:35.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39632025-12-23T22:01:35.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39642025-12-23T22:01:35.349ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39652025-12-23T22:01:35.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39662025-12-23T22:01:35.351ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39672025-12-23T22:01:35.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39682025-12-23T22:01:35.358ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 602.538658ms iteration = 24 status = {}
39692025-12-23T22:01:39.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 140 reason = Timeout
39702025-12-23T22:01:39.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39712025-12-23T22:01:39.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39722025-12-23T22:01:39.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39732025-12-23T22:01:39.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
39742025-12-23T22:01:39.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39752025-12-23T22:01:39.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39762025-12-23T22:01:39.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
39772025-12-23T22:01:39.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.855042ms iteration = 140 status = {"config_updated":false}
39782025-12-23T22:01:41.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 457 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = c7137313-55fa-45cb-b293-ffe6a5c9bf3d response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
39792025-12-23T22:01:44.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
39802025-12-23T22:01:44.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39812025-12-23T22:01:44.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
39822025-12-23T22:01:44.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39832025-12-23T22:01:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39842025-12-23T22:01:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39852025-12-23T22:01:44.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 141 reason = Timeout
39862025-12-23T22:01:44.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39872025-12-23T22:01:44.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39882025-12-23T22:01:44.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39892025-12-23T22:01:44.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39902025-12-23T22:01:44.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39912025-12-23T22:01:44.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39922025-12-23T22:01:44.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.752177ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39932025-12-23T22:01:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39942025-12-23T22:01:44.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39952025-12-23T22:01:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
39962025-12-23T22:01:44.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39972025-12-23T22:01:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39982025-12-23T22:01:44.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39992025-12-23T22:01:44.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
40002025-12-23T22:01:44.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.002857ms iteration = 141 status = {"config_updated":false}
40012025-12-23T22:01:44.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40022025-12-23T22:01:44.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40032025-12-23T22:01:44.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40042025-12-23T22:01:44.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
40052025-12-23T22:01:44.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40062025-12-23T22:01:44.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40072025-12-23T22:01:44.781ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
40082025-12-23T22:01:44.781ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.010285ms iteration = 71 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
40092025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 48 reason = Timeout
40102025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 48 reason = Timeout
40112025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
40122025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40132025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 142 reason = Timeout
40142025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 63 reason = Timeout
40152025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40162025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
40172025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40182025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
40192025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40202025-12-23T22:01:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
40212025-12-23T22:01:49.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40222025-12-23T22:01:49.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40232025-12-23T22:01:49.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40242025-12-23T22:01:49.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40252025-12-23T22:01:49.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40262025-12-23T22:01:49.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40272025-12-23T22:01:49.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
40282025-12-23T22:01:49.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
40292025-12-23T22:01:49.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
40302025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
40312025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
40322025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
40332025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
40342025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
40352025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40362025-12-23T22:01:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40372025-12-23T22:01:49.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.5037ms iteration = 48 status = "NoSitrep"
40382025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40392025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40402025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e original_time_started = 2025-12-23 22:01:06.107605 UTC
40412025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.962893ms iteration = 63 status = {"Loaded":{"collection_id":"7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e","time_loaded":"2025-12-23T22:01:49.756405925Z","time_started":"2025-12-23T22:01:06.107605Z"}}
40422025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
40432025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.21018ms iteration = 142 status = {"config_updated":false}
40442025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40452025-12-23T22:01:49.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40462025-12-23T22:01:49.767ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40472025-12-23T22:01:49.768ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
40482025-12-23T22:01:49.769ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
40492025-12-23T22:01:49.769ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
40502025-12-23T22:01:49.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.45737ms iteration = 48 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
40512025-12-23T22:01:51.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 467 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = e0d2d1ad-2f1f-44eb-a5a7-06d70454e37a response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
40522025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
40532025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
40542025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40552025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40562025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40572025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40582025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 143 reason = Timeout
40592025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40602025-12-23T22:01:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40612025-12-23T22:01:54.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40622025-12-23T22:01:54.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40632025-12-23T22:01:54.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40642025-12-23T22:01:54.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40652025-12-23T22:01:54.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.332604ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40662025-12-23T22:01:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40672025-12-23T22:01:54.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40682025-12-23T22:01:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40692025-12-23T22:01:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
40702025-12-23T22:01:54.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40712025-12-23T22:01:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40722025-12-23T22:01:54.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40732025-12-23T22:01:54.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40742025-12-23T22:01:54.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
40752025-12-23T22:01:54.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.588484ms iteration = 143 status = {"config_updated":false}
40762025-12-23T22:01:54.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40772025-12-23T22:01:54.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
40782025-12-23T22:01:54.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40792025-12-23T22:01:54.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40802025-12-23T22:01:54.778ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
40812025-12-23T22:01:54.778ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.700559ms iteration = 72 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
40822025-12-23T22:01:59.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 144 reason = Timeout
40832025-12-23T22:01:59.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40842025-12-23T22:01:59.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40852025-12-23T22:01:59.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40862025-12-23T22:01:59.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
40872025-12-23T22:01:59.760ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40882025-12-23T22:01:59.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40892025-12-23T22:01:59.762ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
40902025-12-23T22:01:59.762ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.945446ms iteration = 144 status = {"config_updated":false}
40912025-12-23T22:02:01.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = b97307df-242c-4bb4-abd6-336f7a5eaeb5 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
40922025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
40932025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
40942025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
40952025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
40962025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
40972025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40982025-12-23T22:02:04.619ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-23 21:52:04.619821679 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
40992025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
41002025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
41012025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
41022025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
41032025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
41042025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
41052025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
41062025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41072025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
41082025-12-23T22:02:04.619ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
41092025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
41102025-12-23T22:02:04.620ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
41112025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
41122025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
41132025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
41142025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41152025-12-23T22:02:04.620ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = ExternalNetworking step id = Ensure
41162025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41172025-12-23T22:02:04.620ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2bc6e36d-0fab-4746-9740-073e15922b00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 720029842 } })
41182025-12-23T22:02:04.620ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a98c752e-5a58-41fa-9f53-a8d4f529a42f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e9c3ce58-81e4-4140-937f-1b169b78f235.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 720032022 } })
41192025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41202025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
41212025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
41222025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
41232025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41242025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41252025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
41262025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
41272025-12-23T22:02:04.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.235995ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
41282025-12-23T22:02:04.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41292025-12-23T22:02:04.621ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
41302025-12-23T22:02:04.621ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.434371ms iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
41312025-12-23T22:02:04.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41322025-12-23T22:02:04.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41332025-12-23T22:02:04.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41342025-12-23T22:02:04.623ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41352025-12-23T22:02:04.623ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 723111649 } })
41362025-12-23T22:02:04.623ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-23 21:52:04.619821679 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
41372025-12-23T22:02:04.623ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.753736ms iteration = 13 status = {"expiration":"2025-12-23T21:52:04.619821679Z","pruned":[]}
41382025-12-23T22:02:04.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 723560812 } })
41392025-12-23T22:02:04.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41402025-12-23T22:02:04.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41412025-12-23T22:02:04.624ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
41422025-12-23T22:02:04.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41432025-12-23T22:02:04.625ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 724876818 } })
41442025-12-23T22:02:04.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41452025-12-23T22:02:04.628ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41462025-12-23T22:02:04.628ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41472025-12-23T22:02:04.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
41482025-12-23T22:02:04.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41492025-12-23T22:02:04.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41502025-12-23T22:02:04.632ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.76963ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
41512025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
41522025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
41532025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
41542025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
41552025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41562025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41572025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41582025-12-23T22:02:04.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
41592025-12-23T22:02:04.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
41602025-12-23T22:02:04.634ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
41612025-12-23T22:02:04.634ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
41622025-12-23T22:02:04.634ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
41632025-12-23T22:02:04.634ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41642025-12-23T22:02:04.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41652025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41662025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41672025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
41682025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41692025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 30 reason = Timeout
41702025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
41712025-12-23T22:02:04.640ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
41722025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
41732025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
41742025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 43 reason = Timeout
41752025-12-23T22:02:04.640ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
41762025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 17.171\u{b5}s iteration = 43 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
41772025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
41782025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
41792025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
41802025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
41812025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
41822025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
41832025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
41842025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
41852025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
41862025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
41872025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 13 reason = Timeout
41882025-12-23T22:02:04.640ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
41892025-12-23T22:02:04.640ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
41902025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
41912025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
41922025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
41932025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
41942025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
41952025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
41962025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
41972025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
41982025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
41992025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42002025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42012025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42022025-12-23T22:02:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42032025-12-23T22:02:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42042025-12-23T22:02:04.642ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42052025-12-23T22:02:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42062025-12-23T22:02:04.642ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42072025-12-23T22:02:04.642ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42082025-12-23T22:02:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42092025-12-23T22:02:04.643ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42102025-12-23T22:02:04.644ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
42112025-12-23T22:02:04.644ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.332274ms iteration = 13 status = {"by_rx":{},"error":null}
42122025-12-23T22:02:04.644ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42132025-12-23T22:02:04.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42142025-12-23T22:02:04.644ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.429326ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
42152025-12-23T22:02:04.644ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42162025-12-23T22:02:04.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
42172025-12-23T22:02:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
42182025-12-23T22:02:04.649ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42192025-12-23T22:02:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
42202025-12-23T22:02:04.649ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42212025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
42222025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
42232025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42242025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
42252025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42262025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42272025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
42282025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42292025-12-23T22:02:04.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42302025-12-23T22:02:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
42312025-12-23T22:02:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42322025-12-23T22:02:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42332025-12-23T22:02:04.651ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
42342025-12-23T22:02:04.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
42352025-12-23T22:02:04.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
42362025-12-23T22:02:04.654ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
42372025-12-23T22:02:04.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
42382025-12-23T22:02:04.656ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42392025-12-23T22:02:04.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42402025-12-23T22:02:04.656ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42412025-12-23T22:02:04.657ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 16.372372ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"215.0.0","globs_reprocessed":{},"no_receivers":[]}
42422025-12-23T22:02:04.659ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42432025-12-23T22:02:04.660ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
42442025-12-23T22:02:04.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42452025-12-23T22:02:04.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42462025-12-23T22:02:04.660ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42472025-12-23T22:02:04.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42482025-12-23T22:02:04.661ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42492025-12-23T22:02:04.663ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
42502025-12-23T22:02:04.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
42512025-12-23T22:02:04.665ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
42522025-12-23T22:02:04.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
42532025-12-23T22:02:04.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
42542025-12-23T22:02:04.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42552025-12-23T22:02:04.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42562025-12-23T22:02:04.666ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
42572025-12-23T22:02:04.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42582025-12-23T22:02:04.666ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42592025-12-23T22:02:04.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42602025-12-23T22:02:04.667ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42612025-12-23T22:02:04.669ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
42622025-12-23T22:02:04.669ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
42632025-12-23T22:02:04.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
42642025-12-23T22:02:04.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
42652025-12-23T22:02:04.671ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
42662025-12-23T22:02:04.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
42672025-12-23T22:02:04.671ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
42682025-12-23T22:02:04.676ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
42692025-12-23T22:02:04.680ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42702025-12-23T22:02:04.682ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
42712025-12-23T22:02:04.685ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
42722025-12-23T22:02:04.685ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
42732025-12-23T22:02:04.685ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
42742025-12-23T22:02:04.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
42752025-12-23T22:02:04.691ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
42762025-12-23T22:02:04.696ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
42772025-12-23T22:02:04.715ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
42782025-12-23T22:02:04.718ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
42792025-12-23T22:02:04.719ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = SledList step id = Fetch
42802025-12-23T22:02:04.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42812025-12-23T22:02:04.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42822025-12-23T22:02:04.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42832025-12-23T22:02:04.726ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
42842025-12-23T22:02:04.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42852025-12-23T22:02:04.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42862025-12-23T22:02:04.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42872025-12-23T22:02:04.734ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
42882025-12-23T22:02:04.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42892025-12-23T22:02:04.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42902025-12-23T22:02:04.735ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
42912025-12-23T22:02:04.735ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
42922025-12-23T22:02:04.735ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 94.613691ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
42932025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
42942025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
42952025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
42962025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
42972025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
42982025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
42992025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
43002025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
43012025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
43022025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43032025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
43042025-12-23T22:02:04.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43052025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
43062025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43072025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
43082025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
43092025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43102025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43112025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
43122025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 40 reason = Timeout
43132025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.25\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
43142025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
43152025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43162025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43172025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
43182025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
43192025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 37 reason = Timeout
43202025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
43212025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
43222025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43232025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 49 reason = Timeout
43242025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 49 reason = Timeout
43252025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
43262025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 145 reason = Timeout
43272025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
43282025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43292025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 64 reason = Timeout
43302025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43312025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
43322025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43332025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
43342025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
43352025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
43362025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
43372025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 37 reason = Timeout
43382025-12-23T22:02:04.756ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
43392025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43402025-12-23T22:02:04.756ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
43412025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43422025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43432025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
43442025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
43452025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
43462025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43472025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
43482025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43492025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43502025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
43512025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43522025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
43532025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43542025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43552025-12-23T22:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
43562025-12-23T22:02:04.758ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43572025-12-23T22:02:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
43582025-12-23T22:02:04.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
43592025-12-23T22:02:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
43602025-12-23T22:02:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
43612025-12-23T22:02:04.761ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43622025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.697489ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
43632025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43642025-12-23T22:02:04.762ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
43652025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
43662025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43672025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43682025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43692025-12-23T22:02:04.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
43702025-12-23T22:02:04.767ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43712025-12-23T22:02:04.767ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43722025-12-23T22:02:04.767ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
43732025-12-23T22:02:04.767ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
43742025-12-23T22:02:04.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43752025-12-23T22:02:04.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
43762025-12-23T22:02:04.768ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43772025-12-23T22:02:04.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
43782025-12-23T22:02:04.768ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43792025-12-23T22:02:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43802025-12-23T22:02:04.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43812025-12-23T22:02:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43822025-12-23T22:02:04.769ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43832025-12-23T22:02:04.770ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43842025-12-23T22:02:04.772ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43852025-12-23T22:02:04.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
43862025-12-23T22:02:04.773ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43872025-12-23T22:02:04.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43882025-12-23T22:02:04.774ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43892025-12-23T22:02:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43902025-12-23T22:02:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
43912025-12-23T22:02:04.778ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43922025-12-23T22:02:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43932025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
43942025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
43952025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43962025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
43972025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
43982025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43992025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44002025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
44012025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44022025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44032025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44042025-12-23T22:02:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
44052025-12-23T22:02:04.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
44062025-12-23T22:02:04.780ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
44072025-12-23T22:02:04.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
44082025-12-23T22:02:04.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
44092025-12-23T22:02:04.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44102025-12-23T22:02:04.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44112025-12-23T22:02:04.781ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44122025-12-23T22:02:04.783ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44132025-12-23T22:02:04.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
44142025-12-23T22:02:04.783ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44152025-12-23T22:02:04.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
44162025-12-23T22:02:04.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
44172025-12-23T22:02:04.784ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
44182025-12-23T22:02:04.789ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44192025-12-23T22:02:04.789ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
44202025-12-23T22:02:04.789ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44212025-12-23T22:02:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
44222025-12-23T22:02:04.789ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
44232025-12-23T22:02:04.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
44242025-12-23T22:02:04.794ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44252025-12-23T22:02:04.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44262025-12-23T22:02:04.795ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
44272025-12-23T22:02:04.795ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 38.757694ms iteration = 145 status = {"config_updated":false}
44282025-12-23T22:02:04.796ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44292025-12-23T22:02:04.796ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44302025-12-23T22:02:04.796ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 40.515734ms iteration = 49 status = "NoSitrep"
44312025-12-23T22:02:04.796ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44322025-12-23T22:02:04.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44332025-12-23T22:02:04.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
44342025-12-23T22:02:04.798ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 157.98015ms iteration = 6 status = {"nsuccess":5}
44352025-12-23T22:02:04.799ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e original_time_started = 2025-12-23 22:01:06.107605 UTC
44362025-12-23T22:02:04.799ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 42.662065ms iteration = 64 status = {"Loaded":{"collection_id":"7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e","time_loaded":"2025-12-23T22:02:04.756432625Z","time_started":"2025-12-23T22:01:06.107605Z"}}
44372025-12-23T22:02:04.800ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
44382025-12-23T22:02:04.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
44392025-12-23T22:02:04.800ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44402025-12-23T22:02:04.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44412025-12-23T22:02:04.801ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44422025-12-23T22:02:04.802ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 45.729301ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44432025-12-23T22:02:04.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44442025-12-23T22:02:04.803ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44452025-12-23T22:02:04.803ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 903478320 } })
44462025-12-23T22:02:04.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
44472025-12-23T22:02:04.804ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
44482025-12-23T22:02:04.804ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44492025-12-23T22:02:04.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44502025-12-23T22:02:04.804ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 904455367 } })
44512025-12-23T22:02:04.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
44522025-12-23T22:02:04.804ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44532025-12-23T22:02:04.805ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44542025-12-23T22:02:04.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44552025-12-23T22:02:04.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
44562025-12-23T22:02:04.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44572025-12-23T22:02:04.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44582025-12-23T22:02:04.806ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
44592025-12-23T22:02:04.806ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
44602025-12-23T22:02:04.806ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 166.204442ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
44612025-12-23T22:02:04.806ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44622025-12-23T22:02:04.807ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44632025-12-23T22:02:04.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44642025-12-23T22:02:04.807ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
44652025-12-23T22:02:04.808ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 52.841312ms iteration = 25 status = 0
44662025-12-23T22:02:04.808ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
44672025-12-23T22:02:04.808ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
44682025-12-23T22:02:04.808ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-23 21:50:07.200941 UTC dns_group = external generation = 2
44692025-12-23T22:02:04.808ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 188.920613ms iteration = 13 status = {"generation":2}
44702025-12-23T22:02:04.808ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
44712025-12-23T22:02:04.809ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
44722025-12-23T22:02:04.809ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-23 21:50:06.190854 UTC dns_group = internal generation = 1
44732025-12-23T22:02:04.809ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 189.252142ms iteration = 13 status = {"generation":1}
44742025-12-23T22:02:04.810ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
44752025-12-23T22:02:04.810ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
44762025-12-23T22:02:04.810ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 53.759537ms iteration = 49 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
44772025-12-23T22:02:04.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
44782025-12-23T22:02:04.810ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44792025-12-23T22:02:04.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44802025-12-23T22:02:04.814ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
44812025-12-23T22:02:04.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44822025-12-23T22:02:04.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
44832025-12-23T22:02:04.820ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
44842025-12-23T22:02:04.820ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44852025-12-23T22:02:04.821ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "068d51df-2704-4476-8927-a97399373529", "content-length": "26", "date": "Tue, 23 Dec 2025 22:02:04 GMT"} })
44862025-12-23T22:02:04.821ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44872025-12-23T22:02:04.825ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44882025-12-23T22:02:04.826ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1622, tv_nsec: 926191161 } })
44892025-12-23T22:02:04.837ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
44902025-12-23T22:02:04.837ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 81.706727ms iteration = 73 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
44912025-12-23T22:02:04.855ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 98.84684ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
44922025-12-23T22:02:04.860ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44932025-12-23T22:02:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44942025-12-23T22:02:04.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 110.055246ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44952025-12-23T22:02:04.865ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44962025-12-23T22:02:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44972025-12-23T22:02:04.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44982025-12-23T22:02:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44992025-12-23T22:02:04.872ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 116.113517ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
45002025-12-23T22:02:04.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45012025-12-23T22:02:04.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
45022025-12-23T22:02:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45032025-12-23T22:02:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45042025-12-23T22:02:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45052025-12-23T22:02:04.883ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45062025-12-23T22:02:04.883ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
45072025-12-23T22:02:04.883ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 127.626252ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
45082025-12-23T22:02:04.883ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
45092025-12-23T22:02:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
45102025-12-23T22:02:04.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45112025-12-23T22:02:04.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
45122025-12-23T22:02:04.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45132025-12-23T22:02:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45142025-12-23T22:02:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45152025-12-23T22:02:04.888ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45162025-12-23T22:02:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45172025-12-23T22:02:04.890ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
45182025-12-23T22:02:04.892ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45192025-12-23T22:02:04.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45202025-12-23T22:02:04.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45212025-12-23T22:02:04.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45222025-12-23T22:02:04.895ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45232025-12-23T22:02:04.895ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 275.821196ms iteration = 25 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
45242025-12-23T22:02:04.896ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
45252025-12-23T22:02:04.897ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 140.63692ms iteration = 37 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
45262025-12-23T22:02:04.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45272025-12-23T22:02:04.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45282025-12-23T22:02:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45292025-12-23T22:02:04.906ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 149.963033ms iteration = 40 status = {"physical_disks_added":0}
45302025-12-23T22:02:04.907ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-23T21:50:00.171530Z, time_modified: 2025-12-23T21:50:00.171530Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
45312025-12-23T22:02:04.907ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 287.240627ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]}},"default_endpoint":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
45322025-12-23T22:02:04.919ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
45332025-12-23T22:02:04.920ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 163.623479ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
45342025-12-23T22:02:04.920ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45352025-12-23T22:02:04.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45362025-12-23T22:02:04.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45372025-12-23T22:02:04.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45382025-12-23T22:02:04.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45392025-12-23T22:02:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
45402025-12-23T22:02:04.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45412025-12-23T22:02:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45422025-12-23T22:02:04.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45432025-12-23T22:02:04.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45442025-12-23T22:02:04.933ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45452025-12-23T22:02:04.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45462025-12-23T22:02:04.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45472025-12-23T22:02:04.935ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45482025-12-23T22:02:04.939ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
45492025-12-23T22:02:04.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45502025-12-23T22:02:04.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45512025-12-23T22:02:04.941ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45522025-12-23T22:02:04.941ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45532025-12-23T22:02:04.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45542025-12-23T22:02:04.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45552025-12-23T22:02:04.943ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45562025-12-23T22:02:04.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
45572025-12-23T22:02:04.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45582025-12-23T22:02:04.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45592025-12-23T22:02:04.948ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45602025-12-23T22:02:04.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45612025-12-23T22:02:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45622025-12-23T22:02:04.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45632025-12-23T22:02:04.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45642025-12-23T22:02:04.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45652025-12-23T22:02:04.964ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
45662025-12-23T22:02:04.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45672025-12-23T22:02:04.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45682025-12-23T22:02:04.969ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
45692025-12-23T22:02:04.970ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "5258b30d-0cc0-4901-9f5d-d219d1f67d21", "date": "Tue, 23 Dec 2025 22:02:04 GMT"} })
45702025-12-23T22:02:04.970ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
45712025-12-23T22:02:04.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45722025-12-23T22:02:04.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45732025-12-23T22:02:04.972ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
45742025-12-23T22:02:04.972ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 331.560429ms iteration = 13 status = {"errors":[],"probes_by_sled":{"ff85f534-787c-483e-ba89-a37cdea1dd38":0}}
45752025-12-23T22:02:04.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45762025-12-23T22:02:04.974ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
45772025-12-23T22:02:04.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
45782025-12-23T22:02:04.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45792025-12-23T22:02:04.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45802025-12-23T22:02:04.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45812025-12-23T22:02:04.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45822025-12-23T22:02:04.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45832025-12-23T22:02:04.987ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
45842025-12-23T22:02:04.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
45852025-12-23T22:02:04.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45862025-12-23T22:02:04.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45872025-12-23T22:02:04.992ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = DeployNexusRecords step id = Ensure
45882025-12-23T22:02:04.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45892025-12-23T22:02:04.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45902025-12-23T22:02:04.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45912025-12-23T22:02:04.994ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
45922025-12-23T22:02:04.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
45932025-12-23T22:02:04.995ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45942025-12-23T22:02:04.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45952025-12-23T22:02:04.997ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
45962025-12-23T22:02:04.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45972025-12-23T22:02:04.997ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 241.644511ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
45982025-12-23T22:02:04.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45992025-12-23T22:02:04.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46002025-12-23T22:02:04.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46012025-12-23T22:02:05.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46022025-12-23T22:02:05.007ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
46032025-12-23T22:02:05.007ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
46042025-12-23T22:02:05.007ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "74d5cee1-fe72-4007-b574-f998cf17c6c8", "content-length": "372", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
46052025-12-23T22:02:05.007ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46062025-12-23T22:02:05.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46072025-12-23T22:02:05.009ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
46082025-12-23T22:02:05.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46092025-12-23T22:02:05.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46102025-12-23T22:02:05.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46112025-12-23T22:02:05.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46122025-12-23T22:02:05.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46132025-12-23T22:02:05.023ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46142025-12-23T22:02:05.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46152025-12-23T22:02:05.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46162025-12-23T22:02:05.023ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
46172025-12-23T22:02:05.023ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46182025-12-23T22:02:05.024ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "312f680d-367a-42a0-ae3a-5af3b3fbf891", "content-length": "26", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
46192025-12-23T22:02:05.024ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46202025-12-23T22:02:05.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46212025-12-23T22:02:05.027ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
46222025-12-23T22:02:05.028ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
46232025-12-23T22:02:05.028ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
46242025-12-23T22:02:05.028ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f29c2e7b-2708-454b-a414-f445a3bd15fd", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
46252025-12-23T22:02:05.028ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46262025-12-23T22:02:05.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46272025-12-23T22:02:05.029ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 409.953792ms iteration = 25 status = {}
46282025-12-23T22:02:05.031ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46292025-12-23T22:02:05.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46302025-12-23T22:02:05.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46312025-12-23T22:02:05.032ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46322025-12-23T22:02:05.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46332025-12-23T22:02:05.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46342025-12-23T22:02:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46352025-12-23T22:02:05.036ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46362025-12-23T22:02:05.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46372025-12-23T22:02:05.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46382025-12-23T22:02:05.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46392025-12-23T22:02:05.040ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
46402025-12-23T22:02:05.040ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
46412025-12-23T22:02:05.041ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2ad0f73f-3b38-49d4-b8eb-d005f382b2a7", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
46422025-12-23T22:02:05.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46432025-12-23T22:02:05.043ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 286.914139ms iteration = 37 status = {}
46442025-12-23T22:02:05.043ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 38 reason = Dependency
46452025-12-23T22:02:05.043ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
46462025-12-23T22:02:05.043ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46472025-12-23T22:02:05.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46482025-12-23T22:02:05.045ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46492025-12-23T22:02:05.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46502025-12-23T22:02:05.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46512025-12-23T22:02:05.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46522025-12-23T22:02:05.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46532025-12-23T22:02:05.051ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46542025-12-23T22:02:05.051ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46552025-12-23T22:02:05.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46562025-12-23T22:02:05.053ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46572025-12-23T22:02:05.054ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
46582025-12-23T22:02:05.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46592025-12-23T22:02:05.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46602025-12-23T22:02:05.058ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 301.80794ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
46612025-12-23T22:02:05.059ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
46622025-12-23T22:02:05.059ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
46632025-12-23T22:02:05.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
46642025-12-23T22:02:05.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46652025-12-23T22:02:05.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46662025-12-23T22:02:05.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46672025-12-23T22:02:05.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46682025-12-23T22:02:05.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46692025-12-23T22:02:05.069ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
46702025-12-23T22:02:05.079ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = SledAgent step id = Ensure
46712025-12-23T22:02:05.087ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
46722025-12-23T22:02:05.120ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
46732025-12-23T22:02:05.121ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "c8a0f063-a3e4-482e-8f50-101b915db294", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
46742025-12-23T22:02:05.121ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
46752025-12-23T22:02:05.124ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9fb9ec81-f420-424b-bbb9-5b96ad983cf7", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
46762025-12-23T22:02:05.124ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
46772025-12-23T22:02:05.126ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
46782025-12-23T22:02:05.126ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 506.997811ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
46792025-12-23T22:02:05.135ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1623, tv_nsec: 235322707 } })
46802025-12-23T22:02:05.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46812025-12-23T22:02:05.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46822025-12-23T22:02:05.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46832025-12-23T22:02:05.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46842025-12-23T22:02:05.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46852025-12-23T22:02:05.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
46862025-12-23T22:02:05.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46872025-12-23T22:02:05.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46882025-12-23T22:02:05.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46892025-12-23T22:02:05.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46902025-12-23T22:02:05.213ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
46912025-12-23T22:02:05.215ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "02e5d773-942a-44d1-b508-2e02b3fa5fd1", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} }) sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
46922025-12-23T22:02:05.217ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = FirewallRules step id = Ensure
46932025-12-23T22:02:05.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46942025-12-23T22:02:05.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46952025-12-23T22:02:05.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46962025-12-23T22:02:05.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46972025-12-23T22:02:05.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46982025-12-23T22:02:05.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46992025-12-23T22:02:05.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47002025-12-23T22:02:05.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47012025-12-23T22:02:05.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47022025-12-23T22:02:05.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47032025-12-23T22:02:05.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47042025-12-23T22:02:05.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47052025-12-23T22:02:05.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47062025-12-23T22:02:05.233ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
47072025-12-23T22:02:05.233ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
47082025-12-23T22:02:05.233ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "72541ab2-9591-454b-8dac-75f6ed81af8b", "content-length": "372", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
47092025-12-23T22:02:05.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47102025-12-23T22:02:05.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47112025-12-23T22:02:05.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
47122025-12-23T22:02:05.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47132025-12-23T22:02:05.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47142025-12-23T22:02:05.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47152025-12-23T22:02:05.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47162025-12-23T22:02:05.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47172025-12-23T22:02:05.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47182025-12-23T22:02:05.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47192025-12-23T22:02:05.247ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
47202025-12-23T22:02:05.247ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 606.594963ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
47212025-12-23T22:02:05.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
47222025-12-23T22:02:05.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47232025-12-23T22:02:05.253ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
47242025-12-23T22:02:05.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47252025-12-23T22:02:05.253ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "96ab5165-0751-4164-b160-0b06ab3565bb", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47262025-12-23T22:02:05.253ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47272025-12-23T22:02:05.255ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1623, tv_nsec: 354555593 } })
47282025-12-23T22:02:05.255ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "747d0e13-9fcc-49cf-8504-b4a8173304a0", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47292025-12-23T22:02:05.256ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47302025-12-23T22:02:05.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47312025-12-23T22:02:05.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47322025-12-23T22:02:05.258ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "e7361ad0-2b3f-4593-85d1-3b593e30e590", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47332025-12-23T22:02:05.258ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47342025-12-23T22:02:05.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47352025-12-23T22:02:05.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47362025-12-23T22:02:05.260ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
47372025-12-23T22:02:05.260ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 640.982194ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
47382025-12-23T22:02:05.260ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
47392025-12-23T22:02:05.261ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "34935b20-d6cd-4fc3-af9c-42e168bd96d0", "content-length": "691", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
47402025-12-23T22:02:05.261ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
47412025-12-23T22:02:05.261ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
47422025-12-23T22:02:05.261ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
47432025-12-23T22:02:05.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47442025-12-23T22:02:05.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47452025-12-23T22:02:05.263ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 507.262569ms iteration = 25 status = {}
47462025-12-23T22:02:05.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47472025-12-23T22:02:05.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47482025-12-23T22:02:05.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47492025-12-23T22:02:05.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47502025-12-23T22:02:05.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47512025-12-23T22:02:05.269ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
47522025-12-23T22:02:05.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47532025-12-23T22:02:05.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47542025-12-23T22:02:05.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47552025-12-23T22:02:05.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47562025-12-23T22:02:05.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47572025-12-23T22:02:05.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47582025-12-23T22:02:05.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47592025-12-23T22:02:05.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47602025-12-23T22:02:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
47612025-12-23T22:02:05.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47622025-12-23T22:02:05.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47632025-12-23T22:02:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
47642025-12-23T22:02:05.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47652025-12-23T22:02:05.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47662025-12-23T22:02:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47672025-12-23T22:02:05.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47682025-12-23T22:02:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47692025-12-23T22:02:05.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47702025-12-23T22:02:05.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47712025-12-23T22:02:05.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47722025-12-23T22:02:05.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47732025-12-23T22:02:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
47742025-12-23T22:02:05.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47752025-12-23T22:02:05.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47762025-12-23T22:02:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47772025-12-23T22:02:05.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47782025-12-23T22:02:05.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47792025-12-23T22:02:05.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47802025-12-23T22:02:05.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47812025-12-23T22:02:05.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47822025-12-23T22:02:05.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47832025-12-23T22:02:05.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47842025-12-23T22:02:05.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47852025-12-23T22:02:05.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47862025-12-23T22:02:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47872025-12-23T22:02:05.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
47882025-12-23T22:02:05.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47892025-12-23T22:02:05.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47902025-12-23T22:02:05.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47912025-12-23T22:02:05.294ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
47922025-12-23T22:02:05.294ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
47932025-12-23T22:02:05.294ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f1f46111-736a-4905-b459-af89113da1ec parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
47942025-12-23T22:02:05.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47952025-12-23T22:02:05.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47962025-12-23T22:02:05.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47972025-12-23T22:02:05.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47982025-12-23T22:02:05.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47992025-12-23T22:02:05.298ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
48002025-12-23T22:02:05.298ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48012025-12-23T22:02:05.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48022025-12-23T22:02:05.300ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
48032025-12-23T22:02:05.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48042025-12-23T22:02:05.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48052025-12-23T22:02:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
48062025-12-23T22:02:05.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48072025-12-23T22:02:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48082025-12-23T22:02:05.302ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
48092025-12-23T22:02:05.302ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
48102025-12-23T22:02:05.302ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f157e116-2db3-4a69-8dfa-0693daeb6578", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
48112025-12-23T22:02:05.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48122025-12-23T22:02:05.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48132025-12-23T22:02:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48142025-12-23T22:02:05.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48152025-12-23T22:02:05.304ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
48162025-12-23T22:02:05.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48172025-12-23T22:02:05.304ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
48182025-12-23T22:02:05.304ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 663.631103ms iteration = 30 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
48192025-12-23T22:02:05.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48202025-12-23T22:02:05.305ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48212025-12-23T22:02:05.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48222025-12-23T22:02:05.309ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48232025-12-23T22:02:05.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48242025-12-23T22:02:05.312ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
48252025-12-23T22:02:05.312ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
48262025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
48272025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
48282025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
48292025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
48302025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
48312025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
48322025-12-23T22:02:05.313ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
48332025-12-23T22:02:05.313ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48342025-12-23T22:02:05.314ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9b68f358-001a-45d6-9a28-9f5c6ce89bcc", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
48352025-12-23T22:02:05.314ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
48362025-12-23T22:02:05.314ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
48372025-12-23T22:02:05.314ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
48382025-12-23T22:02:05.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
48392025-12-23T22:02:05.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
48402025-12-23T22:02:05.315ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 272.489342ms iteration = 38 status = {}
48412025-12-23T22:02:05.316ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
48422025-12-23T22:02:05.316ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
48432025-12-23T22:02:05.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48442025-12-23T22:02:05.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
48452025-12-23T22:02:05.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48462025-12-23T22:02:05.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48472025-12-23T22:02:05.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48482025-12-23T22:02:05.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48492025-12-23T22:02:05.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48502025-12-23T22:02:05.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48512025-12-23T22:02:05.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48522025-12-23T22:02:05.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48532025-12-23T22:02:05.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48542025-12-23T22:02:05.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48552025-12-23T22:02:05.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48562025-12-23T22:02:05.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48572025-12-23T22:02:05.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48582025-12-23T22:02:05.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48592025-12-23T22:02:05.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48602025-12-23T22:02:05.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48612025-12-23T22:02:05.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48622025-12-23T22:02:05.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
48632025-12-23T22:02:05.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48642025-12-23T22:02:05.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48652025-12-23T22:02:05.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48662025-12-23T22:02:05.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48672025-12-23T22:02:05.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48682025-12-23T22:02:05.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48692025-12-23T22:02:05.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48702025-12-23T22:02:05.401ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
48712025-12-23T22:02:05.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48722025-12-23T22:02:05.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48732025-12-23T22:02:05.409ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
48742025-12-23T22:02:05.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48752025-12-23T22:02:05.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48762025-12-23T22:02:05.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48772025-12-23T22:02:05.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
48782025-12-23T22:02:05.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48792025-12-23T22:02:05.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48802025-12-23T22:02:05.417ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
48812025-12-23T22:02:05.417ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
48822025-12-23T22:02:05.417ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
48832025-12-23T22:02:05.497ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), time_created: 2025-12-23T21:49:57.392343Z, time_modified: 2025-12-23T21:49:57.392343Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
48842025-12-23T22:02:05.497ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
48852025-12-23T22:02:05.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48862025-12-23T22:02:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48872025-12-23T22:02:05.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48882025-12-23T22:02:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48892025-12-23T22:02:05.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48902025-12-23T22:02:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
48912025-12-23T22:02:05.549ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
48922025-12-23T22:02:05.555ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "22670024-a292-4c71-8aa3-d3c6544bed35", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
48932025-12-23T22:02:05.558ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
48942025-12-23T22:02:05.558ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = Dns step id = Ensure
48952025-12-23T22:02:05.558ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
48962025-12-23T22:02:05.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48972025-12-23T22:02:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48982025-12-23T22:02:05.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48992025-12-23T22:02:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49002025-12-23T22:02:05.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49012025-12-23T22:02:05.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49022025-12-23T22:02:05.619ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
49032025-12-23T22:02:05.620ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
49042025-12-23T22:02:05.620ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
49052025-12-23T22:02:05.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49062025-12-23T22:02:05.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49072025-12-23T22:02:05.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49082025-12-23T22:02:05.622ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "18084d0c-889b-461a-8dc3-7837e5705f99", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
49092025-12-23T22:02:05.622ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
49102025-12-23T22:02:05.622ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
49112025-12-23T22:02:05.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49122025-12-23T22:02:05.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49132025-12-23T22:02:05.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49142025-12-23T22:02:05.630ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b38533f5-0682-454a-aff6-8e0070a2c9ad", "content-length": "682", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
49152025-12-23T22:02:05.630ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49162025-12-23T22:02:05.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49172025-12-23T22:02:05.634ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49182025-12-23T22:02:05.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49192025-12-23T22:02:05.639ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49202025-12-23T22:02:05.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49212025-12-23T22:02:05.641ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
49222025-12-23T22:02:05.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
49232025-12-23T22:02:05.653ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 896.804305ms iteration = 25 status = {}
49242025-12-23T22:02:05.683ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
49252025-12-23T22:02:05.683ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
49262025-12-23T22:02:05.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49272025-12-23T22:02:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49282025-12-23T22:02:05.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49292025-12-23T22:02:05.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49302025-12-23T22:02:05.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49312025-12-23T22:02:05.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49322025-12-23T22:02:05.692ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
49332025-12-23T22:02:05.692ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
49342025-12-23T22:02:05.692ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
49352025-12-23T22:02:05.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49362025-12-23T22:02:05.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49372025-12-23T22:02:05.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49382025-12-23T22:02:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49392025-12-23T22:02:05.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49402025-12-23T22:02:05.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49412025-12-23T22:02:05.698ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
49422025-12-23T22:02:05.698ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
49432025-12-23T22:02:05.698ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
49442025-12-23T22:02:05.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49452025-12-23T22:02:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49462025-12-23T22:02:05.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49472025-12-23T22:02:05.703ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49482025-12-23T22:02:05.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49492025-12-23T22:02:05.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49502025-12-23T22:02:05.705ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
49512025-12-23T22:02:05.705ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
49522025-12-23T22:02:05.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49532025-12-23T22:02:05.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49542025-12-23T22:02:05.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49552025-12-23T22:02:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49562025-12-23T22:02:05.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49572025-12-23T22:02:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49582025-12-23T22:02:05.713ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
49592025-12-23T22:02:05.713ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
49602025-12-23T22:02:05.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49612025-12-23T22:02:05.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49622025-12-23T22:02:05.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49632025-12-23T22:02:05.717ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
49642025-12-23T22:02:05.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49652025-12-23T22:02:05.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49662025-12-23T22:02:05.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49672025-12-23T22:02:05.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49682025-12-23T22:02:05.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49692025-12-23T22:02:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49702025-12-23T22:02:05.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49712025-12-23T22:02:05.725ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
49722025-12-23T22:02:05.727ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
49732025-12-23T22:02:05.727ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
49742025-12-23T22:02:05.727ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = OmicronZones step id = Cleanup
49752025-12-23T22:02:05.727ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = OmicronZones step id = Cleanup
49762025-12-23T22:02:05.727ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = PhysicalDisks step id = Cleanup
49772025-12-23T22:02:05.727ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = Clickhouse step id = Ensure
49782025-12-23T22:02:05.727ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = Clickhouse step id = Ensure
49792025-12-23T22:02:05.759ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
49802025-12-23T22:02:05.765ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "c861023d-781a-450b-b150-243ed13a99fd", "date": "Tue, 23 Dec 2025 22:02:05 GMT"} })
49812025-12-23T22:02:05.766ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = SupportBundles step id = Cleanup
49822025-12-23T22:02:05.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49832025-12-23T22:02:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49842025-12-23T22:02:05.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49852025-12-23T22:02:05.770ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
49862025-12-23T22:02:05.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49872025-12-23T22:02:05.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49882025-12-23T22:02:05.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49892025-12-23T22:02:05.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49902025-12-23T22:02:05.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49912025-12-23T22:02:05.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
49922025-12-23T22:02:05.816ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = OmicronZones step id = Cleanup
49932025-12-23T22:02:05.816ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
49942025-12-23T22:02:05.816ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
49952025-12-23T22:02:05.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49962025-12-23T22:02:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49972025-12-23T22:02:05.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49982025-12-23T22:02:05.820ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
49992025-12-23T22:02:05.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50002025-12-23T22:02:05.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50012025-12-23T22:02:05.860ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
50022025-12-23T22:02:05.860ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(02039deb-aee8-405c-80d5-e584bd199fe1 (blueprint), false)
50032025-12-23T22:02:05.860ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = Cockroach step id = Ensure
50042025-12-23T22:02:05.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50052025-12-23T22:02:05.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50062025-12-23T22:02:05.876ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
50072025-12-23T22:02:05.876ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d273a132-59ca-4ad6-b59c-90bbb5c5ad86 step component = MgsUpdates step id = Ensure
50082025-12-23T22:02:05.876ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Dependency
50092025-12-23T22:02:05.876ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 38 reason = Dependency
50102025-12-23T22:02:05.876ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50112025-12-23T22:02:05.877ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50122025-12-23T22:02:05.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
50132025-12-23T22:02:05.877ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.25728942s iteration = 14 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3350,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":99256202,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":99256152,"secs":0}},"event_index":1,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":99269372,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":273327226,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":273327176,"secs":0}},"event_index":2,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":372616118,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":87116290,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":87116250,"secs":0}},"event_index":3,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":459747528,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":137540813,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":137540782,"secs":0}},"event_index":4,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":597304011,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":341656014,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":341655984,"secs":0}},"event_index":5,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":938972985,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":168458425,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":168458385,"secs":0}},"event_index":6,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107448731,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4170,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4140,"secs":0}},"event_index":7,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107463771,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2420,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2380,"secs":0}},"event_index":8,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107468071,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2870,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2840,"secs":0}},"event_index":9,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107472531,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1950,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1910,"secs":0}},"event_index":10,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":107475771,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39467464,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":39467424,"secs":0}},"event_index":11,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":146944305,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49671822,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":49671782,"secs":0}},"event_index":12,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":196627498,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":44116715,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":44116675,"secs":0}},"event_index":13,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":240753874,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16138265,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16138225,"secs":0}},"event_index":14,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":256898139,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5300,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5260,"secs":0}},"event_index":15,"execution_id":"aee2f377-3837-4e6a-93bc-a2ad1bcad995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":256912039,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"02039deb-aee8-405c-80d5-e584bd199fe1"}
50142025-12-23T22:02:05.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50152025-12-23T22:02:05.878ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50162025-12-23T22:02:05.878ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50172025-12-23T22:02:05.881ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
50182025-12-23T22:02:05.881ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50192025-12-23T22:02:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
50202025-12-23T22:02:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
50212025-12-23T22:02:05.882ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50222025-12-23T22:02:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50232025-12-23T22:02:05.919ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50242025-12-23T22:02:05.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50252025-12-23T22:02:05.920ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50262025-12-23T22:02:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
50272025-12-23T22:02:05.923ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
50282025-12-23T22:02:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
50292025-12-23T22:02:05.924ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
50302025-12-23T22:02:05.924ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 47.889022ms iteration = 38 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
50312025-12-23T22:02:05.966ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(f92b2309-ea11-4dbe-892d-2a47b29ba690, 0), (682e7963-a217-487d-9929-59de1d7683fc, 0), (c52dbd67-418f-4c46-9f2c-a5d9fa4bf644, 0), (7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e, 0)] collection_id = f92b2309-ea11-4dbe-892d-2a47b29ba690
50322025-12-23T22:02:05.966ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50332025-12-23T22:02:05.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
50342025-12-23T22:02:06.134ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = f92b2309-ea11-4dbe-892d-2a47b29ba690 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
50352025-12-23T22:02:06.134ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50362025-12-23T22:02:06.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
50372025-12-23T22:02:06.137ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(682e7963-a217-487d-9929-59de1d7683fc, 0), (c52dbd67-418f-4c46-9f2c-a5d9fa4bf644, 0), (7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e, 0)]
50382025-12-23T22:02:06.138ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1624, tv_nsec: 237742453 } })
50392025-12-23T22:02:06.169ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("01b204c0-22af-4353-ad94-4c2c0d1eb01b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70f89219-475c-48c6-a8c8-388723cccdcf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("7b633ce6-0ad9-42aa-8164-bc5c5f301b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("eaafa87a-91ad-4109-aa73-7065c531ed97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1624, tv_nsec: 269161750 } })
50402025-12-23T22:02:06.170ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
50412025-12-23T22:02:06.170ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
50422025-12-23T22:02:06.170ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
50432025-12-23T22:02:06.170ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9513995a-04cc-4911-8ac7-eeaf87884b5e", "content-length": "476", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50442025-12-23T22:02:06.170ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
50452025-12-23T22:02:06.170ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfb733c1-0415-4ec7-a588-c383f435a538", "content-length": "734", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50462025-12-23T22:02:06.171ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
50472025-12-23T22:02:06.171ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "42d3f5ac-e96e-49fa-be04-3d0228decaf6", "content-length": "111", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50482025-12-23T22:02:06.171ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
50492025-12-23T22:02:06.171ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "59220ac9-2a00-463f-9344-4d5fa6f36e6d", "content-length": "111", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50502025-12-23T22:02:06.171ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
50512025-12-23T22:02:06.172ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6d200fd-f332-4226-9423-df70a020e461", "content-length": "163", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50522025-12-23T22:02:06.172ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
50532025-12-23T22:02:06.172ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4241d0e-96ae-499d-b2eb-7f60f637714c", "content-length": "163", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50542025-12-23T22:02:06.172ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
50552025-12-23T22:02:06.173ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01c16aa6-fb2b-4b1e-9ec6-55626b1229ac", "content-length": "166", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50562025-12-23T22:02:06.173ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
50572025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "568ff343-9b75-4d1c-a6cf-fae2440f6dcb", "content-length": "166", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50582025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
50592025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e6612db-cfec-43bc-819a-c7dc5496cbcc", "content-length": "702", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50602025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
50612025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f80aaf58-7db9-4b78-b9ef-03ed6c7f1bda", "content-length": "727", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50622025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
50632025-12-23T22:02:06.174ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d934fdf7-c041-4ddc-9937-0ee7cbcb40ea", "content-length": "729", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50642025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
50652025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "baab6f10-3bf9-4d54-bc29-d429ec3d1d0d", "content-length": "728", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50662025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
50672025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04739ad9-4282-406f-a8cb-b7544f000c4f", "content-length": "722", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50682025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
50692025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dc7335c-4bf0-46f2-9fd4-64ca52947076", "content-length": "10", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50702025-12-23T22:02:06.175ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
50712025-12-23T22:02:06.176ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4eabd06-ad0d-4b1c-91ec-d22f79639795", "content-length": "144", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50722025-12-23T22:02:06.176ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
50732025-12-23T22:02:06.176ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b252eee1-f726-46ff-b82c-ce5b44c591fa", "content-length": "147", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50742025-12-23T22:02:06.176ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
50752025-12-23T22:02:06.177ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "407d491d-69ae-40a6-8c9f-e74ec4ef2e52", "content-length": "109", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50762025-12-23T22:02:06.177ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
50772025-12-23T22:02:06.177ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9453bd0e-7f52-4915-9f2c-44d46eacae95", "content-length": "109", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50782025-12-23T22:02:06.177ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
50792025-12-23T22:02:06.178ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b3ebb5e-4ae5-4997-9933-24872b8142d4", "content-length": "163", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50802025-12-23T22:02:06.178ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
50812025-12-23T22:02:06.178ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af562fc6-ca4f-4cc5-9e0e-6a0cd44a733c", "content-length": "163", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50822025-12-23T22:02:06.178ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
50832025-12-23T22:02:06.179ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf5d6a39-a892-47dd-888f-39fef91cdd95", "content-length": "166", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50842025-12-23T22:02:06.179ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
50852025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc8fece5-5080-4fb4-b106-52e3b1a4de51", "content-length": "166", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50862025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
50872025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b56f15b-b2ec-4194-b9fc-1bf84c2ed779", "content-length": "702", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50882025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
50892025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63d6c262-c992-4460-b1b5-31b6e16fbbe3", "content-length": "727", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50902025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
50912025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf0b87ca-809e-4fd3-873f-60ec878354de", "content-length": "729", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50922025-12-23T22:02:06.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
50932025-12-23T22:02:06.181ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21007c9a-6f4c-435b-8f1e-44f2776abb3b", "content-length": "728", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
50942025-12-23T22:02:06.181ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50952025-12-23T22:02:06.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
50962025-12-23T22:02:06.183ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50972025-12-23T22:02:06.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
50982025-12-23T22:02:06.185ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50992025-12-23T22:02:06.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
51002025-12-23T22:02:06.187ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51012025-12-23T22:02:06.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
51022025-12-23T22:02:06.190ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51032025-12-23T22:02:06.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
51042025-12-23T22:02:06.305ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
51052025-12-23T22:02:06.305ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
51062025-12-23T22:02:06.352ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "13efe89b-efc2-48b8-b727-2786c9fbd18e", "content-length": "95980", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51072025-12-23T22:02:06.355ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
51082025-12-23T22:02:06.355ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
51092025-12-23T22:02:06.355ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
51102025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
51112025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
51122025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
51132025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
51142025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
51152025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "17d9d58c-1074-4dce-8711-8e9671026184", "content-length": "64", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51162025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
51172025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f7a6f50-2248-4e93-a53d-1047ece4d0db", "content-length": "64", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51182025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "528875f6-bd3b-40df-b3ad-482f416c10e2", "content-length": "64", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51192025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a59fca8-4bdf-4872-8f6f-a44b079a1e0b", "content-length": "64", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51202025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
51212025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
51222025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
51232025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "62193e4f-6a73-437f-a028-626f2b5d06fd", "content-length": "64", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51242025-12-23T22:02:06.355ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
51252025-12-23T22:02:06.392ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "00e4106d-cbb3-4766-8058-e6a2b628f35c", "content-length": "417612", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51262025-12-23T22:02:06.393ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "72146ede-611e-4a88-977d-46620db587a1", "content-length": "415434", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51272025-12-23T22:02:06.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "131d0b9f-0979-44fe-90e1-d4cb2cc1c3e8", "content-length": "411200", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51282025-12-23T22:02:06.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6444775c-28ad-4e89-8123-63f94a2adc58", "content-length": "412768", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51292025-12-23T22:02:06.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7334fbfa-40f1-4a0a-88cc-352dab7338b3", "content-length": "430247", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51302025-12-23T22:02:06.398ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
51312025-12-23T22:02:06.398ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
51322025-12-23T22:02:06.398ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
51332025-12-23T22:02:06.398ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
51342025-12-23T22:02:06.398ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
51352025-12-23T22:02:06.428ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c (omicron_zone)
51362025-12-23T22:02:06.428ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
51372025-12-23T22:02:06.431ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "06eb3518-c65c-43a2-86a9-81fa628a552d", "content-length": "128", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51382025-12-23T22:02:06.433ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
51392025-12-23T22:02:06.523ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
51402025-12-23T22:02:06.523ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
51412025-12-23T22:02:06.524ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c81612c2-e85b-4b6d-b185-bf7e02e7bf01", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51422025-12-23T22:02:06.524ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
51432025-12-23T22:02:06.526ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
51442025-12-23T22:02:06.526ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
51452025-12-23T22:02:06.526ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6acf770b-0b0a-47b0-b308-cc977f611dbe", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51462025-12-23T22:02:06.526ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
51472025-12-23T22:02:06.528ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
51482025-12-23T22:02:06.528ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
51492025-12-23T22:02:06.529ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9da4b774-53a7-4504-a6d5-48bf9ddb329c", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:02:06 GMT"} })
51502025-12-23T22:02:06.529ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
51512025-12-23T22:02:06.531ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
51522025-12-23T22:02:06.531ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
51532025-12-23T22:02:06.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51542025-12-23T22:02:06.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
51552025-12-23T22:02:06.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51562025-12-23T22:02:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
51572025-12-23T22:02:06.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51582025-12-23T22:02:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
51592025-12-23T22:02:06.749ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 388fa4f3-ccd9-44a0-8b06-fc205989c366 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
51602025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 388fa4f3-ccd9-44a0-8b06-fc205989c366 time_started = 2025-12-23 22:02:06.170182 UTC
51612025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 65 reason = Dependency
51622025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 44 reason = Dependency
51632025-12-23T22:02:06.749ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
51642025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
51652025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.791\u{b5}s iteration = 44 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
51662025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 872.653123ms iteration = 16 status = {"collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","time_done":"2025-12-23 22:02:06.531478 UTC","time_started":"2025-12-23 22:02:06.170182 UTC"}
51672025-12-23T22:02:06.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
51682025-12-23T22:02:06.751ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51692025-12-23T22:02:06.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
51702025-12-23T22:02:06.755ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
51712025-12-23T22:02:06.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
51722025-12-23T22:02:06.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
51732025-12-23T22:02:06.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
51742025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 31 reason = Dependency
51752025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 57.235545ms iteration = 65 status = {"Loaded":{"collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","time_loaded":"2025-12-23T22:02:06.749590407Z","time_started":"2025-12-23T22:02:06.170182Z"}}
51762025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 45 reason = Dependency
51772025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 19 reason = Dependency
51782025-12-23T22:02:06.806ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
51792025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51802025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 7.81\u{b5}s iteration = 45 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
51812025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 41 reason = Dependency
51822025-12-23T22:02:06.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
51832025-12-23T22:02:06.807ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51842025-12-23T22:02:06.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51852025-12-23T22:02:06.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51862025-12-23T22:02:06.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
51872025-12-23T22:02:06.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51882025-12-23T22:02:06.808ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51892025-12-23T22:02:06.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51902025-12-23T22:02:06.811ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
51912025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51922025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51932025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
51942025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
51952025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
51962025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
51972025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
51982025-12-23T22:02:06.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
51992025-12-23T22:02:06.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52002025-12-23T22:02:06.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52012025-12-23T22:02:06.814ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52022025-12-23T22:02:06.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52032025-12-23T22:02:06.816ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52042025-12-23T22:02:06.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
52052025-12-23T22:02:06.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52062025-12-23T22:02:06.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52072025-12-23T22:02:06.818ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
52082025-12-23T22:02:06.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52092025-12-23T22:02:06.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52102025-12-23T22:02:06.819ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
52112025-12-23T22:02:06.819ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52122025-12-23T22:02:06.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52132025-12-23T22:02:06.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52142025-12-23T22:02:06.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52152025-12-23T22:02:06.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52162025-12-23T22:02:06.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52172025-12-23T22:02:06.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52182025-12-23T22:02:06.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52192025-12-23T22:02:06.822ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
52202025-12-23T22:02:06.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52212025-12-23T22:02:06.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52222025-12-23T22:02:06.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52232025-12-23T22:02:06.824ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
52242025-12-23T22:02:06.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52252025-12-23T22:02:06.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52262025-12-23T22:02:06.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52272025-12-23T22:02:06.827ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
52282025-12-23T22:02:06.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52292025-12-23T22:02:06.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52302025-12-23T22:02:06.828ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
52312025-12-23T22:02:06.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52322025-12-23T22:02:06.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52332025-12-23T22:02:06.830ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
52342025-12-23T22:02:06.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52352025-12-23T22:02:06.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52362025-12-23T22:02:06.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52372025-12-23T22:02:06.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
52382025-12-23T22:02:06.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52392025-12-23T22:02:06.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52402025-12-23T22:02:06.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52412025-12-23T22:02:06.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
52422025-12-23T22:02:06.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
52432025-12-23T22:02:06.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
52442025-12-23T22:02:06.844ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
52452025-12-23T22:02:06.844ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 37.89147ms iteration = 19 status = {"blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","inventory_collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
52462025-12-23T22:02:06.874ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 67.908847ms iteration = 41 status = {"physical_disks_added":0}
52472025-12-23T22:02:06.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52482025-12-23T22:02:06.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52492025-12-23T22:02:06.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52502025-12-23T22:02:06.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52512025-12-23T22:02:06.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52522025-12-23T22:02:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
52532025-12-23T22:02:06.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52542025-12-23T22:02:06.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52552025-12-23T22:02:06.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52562025-12-23T22:02:06.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52572025-12-23T22:02:06.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52582025-12-23T22:02:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
52592025-12-23T22:02:06.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52602025-12-23T22:02:06.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52612025-12-23T22:02:06.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52622025-12-23T22:02:06.900ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
52632025-12-23T22:02:06.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52642025-12-23T22:02:06.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52652025-12-23T22:02:06.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52662025-12-23T22:02:06.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52672025-12-23T22:02:06.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52682025-12-23T22:02:06.906ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
52692025-12-23T22:02:06.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52702025-12-23T22:02:06.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52712025-12-23T22:02:06.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52722025-12-23T22:02:06.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52732025-12-23T22:02:06.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52742025-12-23T22:02:06.913ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
52752025-12-23T22:02:06.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52762025-12-23T22:02:06.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52772025-12-23T22:02:06.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52782025-12-23T22:02:06.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52792025-12-23T22:02:06.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52802025-12-23T22:02:06.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
52812025-12-23T22:02:06.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52822025-12-23T22:02:06.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52832025-12-23T22:02:06.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52842025-12-23T22:02:06.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52852025-12-23T22:02:06.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52862025-12-23T22:02:06.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52872025-12-23T22:02:06.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52882025-12-23T22:02:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
52892025-12-23T22:02:06.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52902025-12-23T22:02:06.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52912025-12-23T22:02:06.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52922025-12-23T22:02:06.948ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
52932025-12-23T22:02:06.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52942025-12-23T22:02:06.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52952025-12-23T22:02:06.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52962025-12-23T22:02:06.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52972025-12-23T22:02:06.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
52982025-12-23T22:02:06.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
52992025-12-23T22:02:06.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53002025-12-23T22:02:06.957ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
53012025-12-23T22:02:06.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53022025-12-23T22:02:06.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53032025-12-23T22:02:06.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53042025-12-23T22:02:06.961ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
53052025-12-23T22:02:06.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53062025-12-23T22:02:06.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53072025-12-23T22:02:06.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53082025-12-23T22:02:06.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53092025-12-23T22:02:06.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53102025-12-23T22:02:06.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53112025-12-23T22:02:06.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53122025-12-23T22:02:06.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
53132025-12-23T22:02:07.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53142025-12-23T22:02:07.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53152025-12-23T22:02:07.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53162025-12-23T22:02:07.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53172025-12-23T22:02:07.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53182025-12-23T22:02:07.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
53192025-12-23T22:02:07.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53202025-12-23T22:02:07.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53212025-12-23T22:02:07.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53222025-12-23T22:02:07.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53232025-12-23T22:02:07.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53242025-12-23T22:02:07.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
53252025-12-23T22:02:07.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53262025-12-23T22:02:07.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53272025-12-23T22:02:07.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53282025-12-23T22:02:07.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53292025-12-23T22:02:07.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53302025-12-23T22:02:07.028ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
53312025-12-23T22:02:07.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53322025-12-23T22:02:07.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53332025-12-23T22:02:07.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53342025-12-23T22:02:07.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53352025-12-23T22:02:07.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53362025-12-23T22:02:07.034ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
53372025-12-23T22:02:07.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53382025-12-23T22:02:07.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53392025-12-23T22:02:07.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53402025-12-23T22:02:07.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53412025-12-23T22:02:07.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53422025-12-23T22:02:07.040ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
53432025-12-23T22:02:07.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53442025-12-23T22:02:07.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53452025-12-23T22:02:07.077ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
53462025-12-23T22:02:07.077ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
53472025-12-23T22:02:07.077ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 4d61a28b-f35a-414f-925f-cd040137cd0b parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
53482025-12-23T22:02:07.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53492025-12-23T22:02:07.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53502025-12-23T22:02:07.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53512025-12-23T22:02:07.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
53522025-12-23T22:02:07.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53532025-12-23T22:02:07.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53542025-12-23T22:02:07.084ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
53552025-12-23T22:02:07.084ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
53562025-12-23T22:02:07.084ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 278.030128ms iteration = 31 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
53572025-12-23T22:02:09.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 146 reason = Timeout
53582025-12-23T22:02:09.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53592025-12-23T22:02:09.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53602025-12-23T22:02:09.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53612025-12-23T22:02:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
53622025-12-23T22:02:09.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53632025-12-23T22:02:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53642025-12-23T22:02:09.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
53652025-12-23T22:02:09.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.133129ms iteration = 146 status = {"config_updated":false}
53662025-12-23T22:02:11.816ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:36589
53672025-12-23T22:02:11.816ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = 49637f38-228d-45db-9822-5f7059b660d7 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending version = 1.0.0
53682025-12-23T22:02:11.816ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = 49637f38-228d-45db-9822-5f7059b660d7 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
53692025-12-23T22:02:11.817ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = 49637f38-228d-45db-9822-5f7059b660d7 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
53702025-12-23T22:02:11.820ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3940 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = 49637f38-228d-45db-9822-5f7059b660d7 response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending
53712025-12-23T22:02:11.820ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = ecbe2bbc-3739-417f-97a6-16104313e712 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D version = 1.0.0
53722025-12-23T22:02:11.820ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = ecbe2bbc-3739-417f-97a6-16104313e712 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
53732025-12-23T22:02:11.821ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = ecbe2bbc-3739-417f-97a6-16104313e712 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
53742025-12-23T22:02:11.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 389 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = b75a60c1-1966-4540-b7b5-edaa0528816d response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
53752025-12-23T22:02:11.823ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2962 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:36589 req_id = ecbe2bbc-3739-417f-97a6-16104313e712 response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D
53762025-12-23T22:02:14.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
53772025-12-23T22:02:14.754ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
53782025-12-23T22:02:14.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53792025-12-23T22:02:14.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53802025-12-23T22:02:14.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53812025-12-23T22:02:14.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53822025-12-23T22:02:14.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 147 reason = Timeout
53832025-12-23T22:02:14.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53842025-12-23T22:02:14.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53852025-12-23T22:02:14.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53862025-12-23T22:02:14.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53872025-12-23T22:02:14.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53882025-12-23T22:02:14.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53892025-12-23T22:02:14.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53902025-12-23T22:02:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
53912025-12-23T22:02:14.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53922025-12-23T22:02:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53932025-12-23T22:02:14.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53942025-12-23T22:02:14.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
53952025-12-23T22:02:14.765ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.107797ms iteration = 147 status = {"config_updated":false}
53962025-12-23T22:02:14.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53972025-12-23T22:02:14.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53982025-12-23T22:02:14.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
53992025-12-23T22:02:14.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54002025-12-23T22:02:14.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54012025-12-23T22:02:14.786ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
54022025-12-23T22:02:14.786ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.599822ms iteration = 74 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
54032025-12-23T22:02:14.798ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54042025-12-23T22:02:14.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54052025-12-23T22:02:14.849ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 94.825787ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54062025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 50 reason = Timeout
54072025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 50 reason = Timeout
54082025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54092025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
54102025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 66 reason = Timeout
54112025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
54122025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 148 reason = Timeout
54132025-12-23T22:02:19.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
54142025-12-23T22:02:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
54152025-12-23T22:02:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54162025-12-23T22:02:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
54172025-12-23T22:02:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
54182025-12-23T22:02:19.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54192025-12-23T22:02:19.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54202025-12-23T22:02:19.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54212025-12-23T22:02:19.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54222025-12-23T22:02:19.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54232025-12-23T22:02:19.759ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54242025-12-23T22:02:19.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
54252025-12-23T22:02:19.763ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
54262025-12-23T22:02:19.763ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
54272025-12-23T22:02:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
54282025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
54292025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
54302025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
54312025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
54322025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54332025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
54342025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
54352025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54362025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.692137ms iteration = 50 status = "NoSitrep"
54372025-12-23T22:02:19.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54382025-12-23T22:02:19.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54392025-12-23T22:02:19.765ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 388fa4f3-ccd9-44a0-8b06-fc205989c366 original_time_started = 2025-12-23 22:02:06.170182 UTC
54402025-12-23T22:02:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.927742ms iteration = 66 status = {"Loaded":{"collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","time_loaded":"2025-12-23T22:02:19.756750773Z","time_started":"2025-12-23T22:02:06.170182Z"}}
54412025-12-23T22:02:19.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
54422025-12-23T22:02:19.766ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.233771ms iteration = 148 status = {"config_updated":false}
54432025-12-23T22:02:19.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54442025-12-23T22:02:19.767ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
54452025-12-23T22:02:19.769ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
54462025-12-23T22:02:19.769ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
54472025-12-23T22:02:19.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.968646ms iteration = 50 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
54482025-12-23T22:02:21.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 495 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 80a01e78-fbf5-44da-bee0-c625667cb9b5 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
54492025-12-23T22:02:24.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
54502025-12-23T22:02:24.754ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
54512025-12-23T22:02:24.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54522025-12-23T22:02:24.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54532025-12-23T22:02:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54542025-12-23T22:02:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54552025-12-23T22:02:24.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 149 reason = Timeout
54562025-12-23T22:02:24.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
54572025-12-23T22:02:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
54582025-12-23T22:02:24.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54592025-12-23T22:02:24.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54602025-12-23T22:02:24.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54612025-12-23T22:02:24.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54622025-12-23T22:02:24.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54632025-12-23T22:02:24.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54642025-12-23T22:02:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54652025-12-23T22:02:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
54662025-12-23T22:02:24.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
54672025-12-23T22:02:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
54682025-12-23T22:02:24.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54692025-12-23T22:02:24.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
54702025-12-23T22:02:24.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.942694ms iteration = 149 status = {"config_updated":false}
54712025-12-23T22:02:24.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
54722025-12-23T22:02:24.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54732025-12-23T22:02:24.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
54742025-12-23T22:02:24.782ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
54752025-12-23T22:02:24.782ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.764254ms iteration = 75 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
54762025-12-23T22:02:24.791ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54772025-12-23T22:02:24.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54782025-12-23T22:02:24.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.111774ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54792025-12-23T22:02:29.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 150 reason = Timeout
54802025-12-23T22:02:29.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
54812025-12-23T22:02:29.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
54822025-12-23T22:02:29.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54832025-12-23T22:02:29.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
54842025-12-23T22:02:29.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
54852025-12-23T22:02:29.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
54862025-12-23T22:02:29.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
54872025-12-23T22:02:29.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.037068ms iteration = 150 status = {"config_updated":false}
54882025-12-23T22:02:31.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 706 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = c34d29af-4600-4653-b0f8-6bcceec1348d response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
54892025-12-23T22:02:34.620ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
54902025-12-23T22:02:34.620ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
54912025-12-23T22:02:34.620ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
54922025-12-23T22:02:34.620ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
54932025-12-23T22:02:34.620ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
54942025-12-23T22:02:34.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
54952025-12-23T22:02:34.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
54962025-12-23T22:02:34.623ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 722823371 } })
54972025-12-23T22:02:34.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 722867882 } })
54982025-12-23T22:02:34.623ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
54992025-12-23T22:02:34.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 723487969 } })
55002025-12-23T22:02:34.624ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.725356ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
55012025-12-23T22:02:34.690ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
55022025-12-23T22:02:34.690ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
55032025-12-23T22:02:34.690ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e34f214-4f86-4672-85fb-feb47e294501", "content-length": "26", "date": "Tue, 23 Dec 2025 22:02:34 GMT"} })
55042025-12-23T22:02:34.690ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
55052025-12-23T22:02:34.692ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
55062025-12-23T22:02:34.692ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 792529468 } })
55072025-12-23T22:02:34.695ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
55082025-12-23T22:02:34.696ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 75.602934ms iteration = 26 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
55092025-12-23T22:02:34.734ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 114.0854ms iteration = 26 status = {}
55102025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
55112025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
55122025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
55132025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
55142025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
55152025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55162025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
55172025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
55182025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
55192025-12-23T22:02:34.754ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
55202025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
55212025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55222025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55232025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
55242025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55252025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55262025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55272025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
55282025-12-23T22:02:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
55292025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
55302025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.58\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
55312025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
55322025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
55332025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
55342025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
55352025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
55362025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 51 reason = Timeout
55372025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 42 reason = Timeout
55382025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
55392025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55402025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 39 reason = Timeout
55412025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 51 reason = Timeout
55422025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
55432025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
55442025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 39 reason = Timeout
55452025-12-23T22:02:34.756ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
55462025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
55472025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
55482025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
55492025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 67 reason = Timeout
55502025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
55512025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
55522025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
55532025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
55542025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55552025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
55562025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
55572025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55582025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 151 reason = Timeout
55592025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55602025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
55612025-12-23T22:02:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
55622025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55632025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
55642025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
55652025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55662025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
55672025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
55682025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
55692025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
55702025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
55712025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
55722025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55732025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55742025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55752025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
55762025-12-23T22:02:34.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55772025-12-23T22:02:34.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
55782025-12-23T22:02:34.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
55792025-12-23T22:02:34.758ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55802025-12-23T22:02:34.759ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55812025-12-23T22:02:34.759ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55822025-12-23T22:02:34.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55832025-12-23T22:02:34.760ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.254138ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55842025-12-23T22:02:34.761ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55852025-12-23T22:02:34.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
55862025-12-23T22:02:34.763ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
55872025-12-23T22:02:34.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
55882025-12-23T22:02:34.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55892025-12-23T22:02:34.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55902025-12-23T22:02:34.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55912025-12-23T22:02:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
55922025-12-23T22:02:34.765ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55932025-12-23T22:02:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55942025-12-23T22:02:34.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55952025-12-23T22:02:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55962025-12-23T22:02:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
55972025-12-23T22:02:34.768ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
55982025-12-23T22:02:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
55992025-12-23T22:02:34.769ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56002025-12-23T22:02:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56012025-12-23T22:02:34.770ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56022025-12-23T22:02:34.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
56032025-12-23T22:02:34.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56042025-12-23T22:02:34.771ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
56052025-12-23T22:02:34.771ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.547121ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
56062025-12-23T22:02:34.771ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.413334ms iteration = 26 status = 0
56072025-12-23T22:02:34.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56082025-12-23T22:02:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
56092025-12-23T22:02:34.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56102025-12-23T22:02:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56112025-12-23T22:02:34.777ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56122025-12-23T22:02:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56132025-12-23T22:02:34.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56142025-12-23T22:02:34.815ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
56152025-12-23T22:02:34.815ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 60.944551ms iteration = 76 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
56162025-12-23T22:02:34.817ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
56172025-12-23T22:02:34.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
56182025-12-23T22:02:34.819ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56192025-12-23T22:02:34.819ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 62.794492ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
56202025-12-23T22:02:34.820ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56212025-12-23T22:02:34.822ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
56222025-12-23T22:02:34.822ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
56232025-12-23T22:02:34.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
56242025-12-23T22:02:34.822ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56252025-12-23T22:02:34.822ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
56262025-12-23T22:02:34.824ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 67.811754ms iteration = 51 status = "NoSitrep"
56272025-12-23T22:02:34.825ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
56282025-12-23T22:02:34.825ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56292025-12-23T22:02:34.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56302025-12-23T22:02:34.828ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
56312025-12-23T22:02:34.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
56322025-12-23T22:02:34.829ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56332025-12-23T22:02:34.829ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
56342025-12-23T22:02:34.829ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56352025-12-23T22:02:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
56362025-12-23T22:02:34.829ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
56372025-12-23T22:02:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
56382025-12-23T22:02:34.831ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56392025-12-23T22:02:34.832ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 931681156 } })
56402025-12-23T22:02:34.832ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
56412025-12-23T22:02:34.832ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.094698ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
56422025-12-23T22:02:34.832ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1652, tv_nsec: 932537820 } })
56432025-12-23T22:02:34.833ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56442025-12-23T22:02:34.833ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.745723ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
56452025-12-23T22:02:34.833ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56462025-12-23T22:02:34.834ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56472025-12-23T22:02:34.834ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
56482025-12-23T22:02:34.834ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
56492025-12-23T22:02:34.834ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56502025-12-23T22:02:34.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
56512025-12-23T22:02:34.835ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
56522025-12-23T22:02:34.835ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
56532025-12-23T22:02:34.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
56542025-12-23T22:02:34.836ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
56552025-12-23T22:02:34.836ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56562025-12-23T22:02:34.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56572025-12-23T22:02:34.837ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 82.709045ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56582025-12-23T22:02:34.837ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56592025-12-23T22:02:34.837ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56602025-12-23T22:02:34.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56612025-12-23T22:02:34.839ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56622025-12-23T22:02:34.839ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56632025-12-23T22:02:34.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
56642025-12-23T22:02:34.839ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
56652025-12-23T22:02:34.839ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
56662025-12-23T22:02:34.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
56672025-12-23T22:02:34.840ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
56682025-12-23T22:02:34.840ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 83.392764ms iteration = 51 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
56692025-12-23T22:02:34.843ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 388fa4f3-ccd9-44a0-8b06-fc205989c366 original_time_started = 2025-12-23 22:02:06.170182 UTC
56702025-12-23T22:02:34.843ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 86.480251ms iteration = 67 status = {"Loaded":{"collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","time_loaded":"2025-12-23T22:02:34.756691342Z","time_started":"2025-12-23T22:02:06.170182Z"}}
56712025-12-23T22:02:34.844ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
56722025-12-23T22:02:34.844ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
56732025-12-23T22:02:34.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56742025-12-23T22:02:34.844ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56752025-12-23T22:02:34.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
56762025-12-23T22:02:34.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56772025-12-23T22:02:34.845ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
56782025-12-23T22:02:34.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
56792025-12-23T22:02:34.847ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
56802025-12-23T22:02:34.847ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 90.265748ms iteration = 151 status = {"config_updated":false}
56812025-12-23T22:02:34.864ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 107.736722ms iteration = 42 status = {"physical_disks_added":0}
56822025-12-23T22:02:34.867ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
56832025-12-23T22:02:34.867ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
56842025-12-23T22:02:34.868ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da18fd63-10d3-464d-982c-eaf47a878998", "content-length": "26", "date": "Tue, 23 Dec 2025 22:02:34 GMT"} })
56852025-12-23T22:02:34.868ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
56862025-12-23T22:02:34.869ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
56872025-12-23T22:02:34.872ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
56882025-12-23T22:02:34.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
56892025-12-23T22:02:34.877ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
56902025-12-23T22:02:34.877ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
56912025-12-23T22:02:34.877ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 123.041194ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
56922025-12-23T22:02:34.884ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56932025-12-23T22:02:34.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56942025-12-23T22:02:34.886ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56952025-12-23T22:02:34.886ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 129.865186ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56962025-12-23T22:02:34.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56972025-12-23T22:02:34.890ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56982025-12-23T22:02:34.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56992025-12-23T22:02:34.900ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1653, tv_nsec: 477098 } })
57002025-12-23T22:02:34.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 164.809113ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
57012025-12-23T22:02:34.938ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
57022025-12-23T22:02:34.938ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 181.649007ms iteration = 39 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
57032025-12-23T22:02:34.943ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57042025-12-23T22:02:34.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
57052025-12-23T22:02:34.945ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57062025-12-23T22:02:34.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57072025-12-23T22:02:34.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57082025-12-23T22:02:34.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
57092025-12-23T22:02:34.948ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1653, tv_nsec: 48180804 } })
57102025-12-23T22:02:34.950ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57112025-12-23T22:02:34.952ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
57122025-12-23T22:02:34.952ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57132025-12-23T22:02:34.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
57142025-12-23T22:02:35.051ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57152025-12-23T22:02:35.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57162025-12-23T22:02:35.054ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
57172025-12-23T22:02:35.054ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57182025-12-23T22:02:35.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57192025-12-23T22:02:35.067ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
57202025-12-23T22:02:35.067ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
57212025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
57222025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
57232025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
57242025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
57252025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
57262025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
57272025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
57282025-12-23T22:02:35.068ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
57292025-12-23T22:02:35.069ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
57302025-12-23T22:02:35.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57312025-12-23T22:02:35.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57322025-12-23T22:02:35.071ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
57332025-12-23T22:02:35.071ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
57342025-12-23T22:02:35.080ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
57352025-12-23T22:02:35.080ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57362025-12-23T22:02:35.080ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce9a644d-6d7c-419c-8822-97c164b5e81c", "content-length": "372", "date": "Tue, 23 Dec 2025 22:02:35 GMT"} })
57372025-12-23T22:02:35.080ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57382025-12-23T22:02:35.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57392025-12-23T22:02:35.088ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
57402025-12-23T22:02:35.091ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7196e541-ab16-4cb6-9737-6740f6c72051", "content-length": "691", "date": "Tue, 23 Dec 2025 22:02:35 GMT"} })
57412025-12-23T22:02:35.091ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
57422025-12-23T22:02:35.091ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
57432025-12-23T22:02:35.091ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
57442025-12-23T22:02:35.093ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 336.5611ms iteration = 26 status = {}
57452025-12-23T22:02:35.094ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
57462025-12-23T22:02:35.094ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
57472025-12-23T22:02:35.094ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bc965349-ac64-4eba-9856-46e7e12c2736", "date": "Tue, 23 Dec 2025 22:02:35 GMT"} })
57482025-12-23T22:02:35.094ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57492025-12-23T22:02:35.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57502025-12-23T22:02:35.097ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57512025-12-23T22:02:35.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57522025-12-23T22:02:35.100ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57532025-12-23T22:02:35.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57542025-12-23T22:02:35.103ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
57552025-12-23T22:02:35.103ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
57562025-12-23T22:02:35.104ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2a83fe9a-e522-4e14-9789-d08c5285c72b", "date": "Tue, 23 Dec 2025 22:02:35 GMT"} })
57572025-12-23T22:02:35.106ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 349.821954ms iteration = 39 status = {}
57582025-12-23T22:02:35.332ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1c7bedce-410f-494a-b51e-c1572253d9cc", "date": "Tue, 23 Dec 2025 22:02:35 GMT"} })
57592025-12-23T22:02:35.332ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
57602025-12-23T22:02:35.332ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
57612025-12-23T22:02:35.332ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "33054999-d326-4f29-b054-1b0cac702b89", "content-length": "682", "date": "Tue, 23 Dec 2025 22:02:35 GMT"} })
57622025-12-23T22:02:35.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57632025-12-23T22:02:35.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57642025-12-23T22:02:35.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57652025-12-23T22:02:35.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57662025-12-23T22:02:35.340ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57672025-12-23T22:02:35.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57682025-12-23T22:02:35.342ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57692025-12-23T22:02:35.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57702025-12-23T22:02:35.349ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 592.687241ms iteration = 26 status = {}
57712025-12-23T22:02:39.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 152 reason = Timeout
57722025-12-23T22:02:39.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57732025-12-23T22:02:39.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57742025-12-23T22:02:39.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57752025-12-23T22:02:39.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
57762025-12-23T22:02:39.760ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57772025-12-23T22:02:39.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57782025-12-23T22:02:39.762ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
57792025-12-23T22:02:39.762ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.051709ms iteration = 152 status = {"config_updated":false}
57802025-12-23T22:02:41.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 451 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = e5587c72-e5d6-4c12-8df5-3cd0f2c0526f response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
57812025-12-23T22:02:44.754ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
57822025-12-23T22:02:44.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57832025-12-23T22:02:44.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
57842025-12-23T22:02:44.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57852025-12-23T22:02:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57862025-12-23T22:02:44.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57872025-12-23T22:02:44.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 153 reason = Timeout
57882025-12-23T22:02:44.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57892025-12-23T22:02:44.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57902025-12-23T22:02:44.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57912025-12-23T22:02:44.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57922025-12-23T22:02:44.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57932025-12-23T22:02:44.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57942025-12-23T22:02:44.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.609572ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
57952025-12-23T22:02:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57962025-12-23T22:02:44.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57972025-12-23T22:02:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57982025-12-23T22:02:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
57992025-12-23T22:02:44.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58002025-12-23T22:02:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58012025-12-23T22:02:44.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58022025-12-23T22:02:44.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58032025-12-23T22:02:44.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
58042025-12-23T22:02:44.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.254945ms iteration = 153 status = {"config_updated":false}
58052025-12-23T22:02:44.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58062025-12-23T22:02:44.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
58072025-12-23T22:02:44.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58082025-12-23T22:02:44.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58092025-12-23T22:02:44.783ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
58102025-12-23T22:02:44.783ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.918436ms iteration = 77 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
58112025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 52 reason = Timeout
58122025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
58132025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 68 reason = Timeout
58142025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
58152025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 52 reason = Timeout
58162025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 154 reason = Timeout
58172025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58182025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58192025-12-23T22:02:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
58202025-12-23T22:02:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
58212025-12-23T22:02:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58222025-12-23T22:02:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58232025-12-23T22:02:49.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58242025-12-23T22:02:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58252025-12-23T22:02:49.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58262025-12-23T22:02:49.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58272025-12-23T22:02:49.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58282025-12-23T22:02:49.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58292025-12-23T22:02:49.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
58302025-12-23T22:02:49.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
58312025-12-23T22:02:49.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
58322025-12-23T22:02:49.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
58332025-12-23T22:02:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
58342025-12-23T22:02:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
58352025-12-23T22:02:49.763ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58362025-12-23T22:02:49.763ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
58372025-12-23T22:02:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
58382025-12-23T22:02:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58392025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.556184ms iteration = 52 status = "NoSitrep"
58402025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58412025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58422025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
58432025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.243663ms iteration = 154 status = {"config_updated":false}
58442025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 388fa4f3-ccd9-44a0-8b06-fc205989c366 original_time_started = 2025-12-23 22:02:06.170182 UTC
58452025-12-23T22:02:49.764ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.297314ms iteration = 68 status = {"Loaded":{"collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","time_loaded":"2025-12-23T22:02:49.756563418Z","time_started":"2025-12-23T22:02:06.170182Z"}}
58462025-12-23T22:02:49.765ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58472025-12-23T22:02:49.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58482025-12-23T22:02:49.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58492025-12-23T22:02:49.768ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
58502025-12-23T22:02:49.769ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
58512025-12-23T22:02:49.769ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
58522025-12-23T22:02:49.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.337067ms iteration = 52 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
58532025-12-23T22:02:51.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 435 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 0217dd1a-5f98-4269-bd6d-410af431578d response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
58542025-12-23T22:02:54.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
58552025-12-23T22:02:54.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58562025-12-23T22:02:54.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
58572025-12-23T22:02:54.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58582025-12-23T22:02:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58592025-12-23T22:02:54.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58602025-12-23T22:02:54.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 155 reason = Timeout
58612025-12-23T22:02:54.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58622025-12-23T22:02:54.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58632025-12-23T22:02:54.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58642025-12-23T22:02:54.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58652025-12-23T22:02:54.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58662025-12-23T22:02:54.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.256132ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58672025-12-23T22:02:54.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58682025-12-23T22:02:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58692025-12-23T22:02:54.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58702025-12-23T22:02:54.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58712025-12-23T22:02:54.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
58722025-12-23T22:02:54.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58732025-12-23T22:02:54.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58742025-12-23T22:02:54.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58752025-12-23T22:02:54.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58762025-12-23T22:02:54.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
58772025-12-23T22:02:54.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.262015ms iteration = 155 status = {"config_updated":false}
58782025-12-23T22:02:54.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58792025-12-23T22:02:54.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
58802025-12-23T22:02:54.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58812025-12-23T22:02:54.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58822025-12-23T22:02:54.778ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
58832025-12-23T22:02:54.778ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.138273ms iteration = 78 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
58842025-12-23T22:02:59.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 156 reason = Timeout
58852025-12-23T22:02:59.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58862025-12-23T22:02:59.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58872025-12-23T22:02:59.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58882025-12-23T22:02:59.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
58892025-12-23T22:02:59.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58902025-12-23T22:02:59.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58912025-12-23T22:02:59.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
58922025-12-23T22:02:59.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.878423ms iteration = 156 status = {"config_updated":false}
58932025-12-23T22:03:01.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 496 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = e6699d57-bd47-4412-95ca-6a0078b47fb8 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
58942025-12-23T22:03:04.618ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
58952025-12-23T22:03:04.618ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
58962025-12-23T22:03:04.618ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
58972025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
58982025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
58992025-12-23T22:03:04.619ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2bc6e36d-0fab-4746-9740-073e15922b00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e34ab301-3b64-4994-9379-711e8a6d4317.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 719058684 } })
59002025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59012025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
59022025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
59032025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
59042025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
59052025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59062025-12-23T22:03:04.619ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-23 21:53:04.619748137 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
59072025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
59082025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
59092025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
59102025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
59112025-12-23T22:03:04.619ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a98c752e-5a58-41fa-9f53-a8d4f529a42f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e9c3ce58-81e4-4140-937f-1b169b78f235.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("f456c4c8-e5fd-45ce-a34c-a01d88dd14ad.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 719372013 } })
59122025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
59132025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59142025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
59152025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
59162025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
59172025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
59182025-12-23T22:03:04.619ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
59192025-12-23T22:03:04.620ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
59202025-12-23T22:03:04.620ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = ExternalNetworking step id = Ensure
59212025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59222025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
59232025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.259096ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
59242025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59252025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
59262025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
59272025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59282025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
59292025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59302025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
59312025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
59322025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.604475ms iteration = 14 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
59332025-12-23T22:03:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59342025-12-23T22:03:04.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59352025-12-23T22:03:04.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59362025-12-23T22:03:04.623ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 722701357 } })
59372025-12-23T22:03:04.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59382025-12-23T22:03:04.623ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-23 21:53:04.619748137 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
59392025-12-23T22:03:04.623ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.663163ms iteration = 14 status = {"expiration":"2025-12-23T21:53:04.619748137Z","pruned":[]}
59402025-12-23T22:03:04.623ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
59412025-12-23T22:03:04.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59422025-12-23T22:03:04.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 723363076 } })
59432025-12-23T22:03:04.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 723375356 } })
59442025-12-23T22:03:04.623ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59452025-12-23T22:03:04.624ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.541149ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
59462025-12-23T22:03:04.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59472025-12-23T22:03:04.625ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59482025-12-23T22:03:04.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
59492025-12-23T22:03:04.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
59502025-12-23T22:03:04.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59512025-12-23T22:03:04.630ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
59522025-12-23T22:03:04.630ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59532025-12-23T22:03:04.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
59542025-12-23T22:03:04.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
59552025-12-23T22:03:04.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
59562025-12-23T22:03:04.633ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
59572025-12-23T22:03:04.633ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
59582025-12-23T22:03:04.633ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
59592025-12-23T22:03:04.633ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59602025-12-23T22:03:04.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59612025-12-23T22:03:04.633ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59622025-12-23T22:03:04.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
59632025-12-23T22:03:04.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59642025-12-23T22:03:04.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
59652025-12-23T22:03:04.636ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59662025-12-23T22:03:04.639ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
59672025-12-23T22:03:04.640ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
59682025-12-23T22:03:04.640ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
59692025-12-23T22:03:04.640ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
59702025-12-23T22:03:04.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
59712025-12-23T22:03:04.640ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 32 reason = Timeout
59722025-12-23T22:03:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
59732025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
59742025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 46 reason = Timeout
59752025-12-23T22:03:04.641ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
59762025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 18.101\u{b5}s iteration = 46 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
59772025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
59782025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
59792025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
59802025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
59812025-12-23T22:03:04.641ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
59822025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
59832025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
59842025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
59852025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
59862025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
59872025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
59882025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
59892025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
59902025-12-23T22:03:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
59912025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
59922025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 14 reason = Timeout
59932025-12-23T22:03:04.643ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
59942025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
59952025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
59962025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
59972025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
59982025-12-23T22:03:04.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
59992025-12-23T22:03:04.644ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60002025-12-23T22:03:04.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
60012025-12-23T22:03:04.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60022025-12-23T22:03:04.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60032025-12-23T22:03:04.646ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
60042025-12-23T22:03:04.646ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.555359ms iteration = 14 status = {"by_rx":{},"error":null}
60052025-12-23T22:03:04.646ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60062025-12-23T22:03:04.646ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: bbfc9dcf-82aa-4f85-adec-55604e2ebfb8 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 2bc6e36d-0fab-4746-9740-073e15922b00 zone_kind = external_dns
60072025-12-23T22:03:04.646ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.638211ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
60082025-12-23T22:03:04.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60092025-12-23T22:03:04.648ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
60102025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
60112025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
60122025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
60132025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
60142025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60152025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
60162025-12-23T22:03:04.649ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
60172025-12-23T22:03:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
60182025-12-23T22:03:04.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60192025-12-23T22:03:04.651ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
60202025-12-23T22:03:04.651ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
60212025-12-23T22:03:04.651ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 8.401477ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
60222025-12-23T22:03:04.651ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: b895f594-3bdf-453b-90ef-a1e1860e79b3 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 } zone_id = 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd zone_kind = nexus
60232025-12-23T22:03:04.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
60242025-12-23T22:03:04.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60252025-12-23T22:03:04.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60262025-12-23T22:03:04.652ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60272025-12-23T22:03:04.652ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 10.209708ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"215.0.0","globs_reprocessed":{},"no_receivers":[]}
60282025-12-23T22:03:04.653ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
60292025-12-23T22:03:04.653ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
60302025-12-23T22:03:04.653ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 13.211093ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
60312025-12-23T22:03:04.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
60322025-12-23T22:03:04.653ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
60332025-12-23T22:03:04.653ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
60342025-12-23T22:03:04.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60352025-12-23T22:03:04.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
60362025-12-23T22:03:04.655ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: 02ca54a1-df2f-476b-b370-5bb7cf9e5af1 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: 3041de85-2a7b-4695-be31-5cba35bc4130, kind: Service { id: 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c }, name: Name("ntp-5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 252, 49])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c zone_kind = boundary_ntp
60372025-12-23T22:03:04.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
60382025-12-23T22:03:04.658ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
60392025-12-23T22:03:04.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
60402025-12-23T22:03:04.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60412025-12-23T22:03:04.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
60422025-12-23T22:03:04.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60432025-12-23T22:03:04.667ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
60442025-12-23T22:03:04.667ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60452025-12-23T22:03:04.670ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
60462025-12-23T22:03:04.670ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
60472025-12-23T22:03:04.670ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-23 21:50:06.190854 UTC dns_group = internal generation = 1
60482025-12-23T22:03:04.670ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 51.849543ms iteration = 14 status = {"generation":1}
60492025-12-23T22:03:04.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
60502025-12-23T22:03:04.671ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
60512025-12-23T22:03:04.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
60522025-12-23T22:03:04.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60532025-12-23T22:03:04.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60542025-12-23T22:03:04.675ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9c3c617-343d-4ede-a9da-2c866e07f669 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 zone_kind = nexus
60552025-12-23T22:03:04.676ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
60562025-12-23T22:03:04.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
60572025-12-23T22:03:04.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60582025-12-23T22:03:04.679ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
60592025-12-23T22:03:04.679ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60602025-12-23T22:03:04.683ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
60612025-12-23T22:03:04.683ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
60622025-12-23T22:03:04.683ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
60632025-12-23T22:03:04.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60642025-12-23T22:03:04.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
60652025-12-23T22:03:04.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60662025-12-23T22:03:04.684ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 4f864a0d-be33-4aeb-b832-f249a7a8d15f (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 } zone_id = a981c0ae-aa35-4460-b263-22312db6f302 zone_kind = nexus
60672025-12-23T22:03:04.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60682025-12-23T22:03:04.690ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
60692025-12-23T22:03:04.692ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: a9f1db65-825f-4f40-8160-672b29924e42 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 } zone_id = e34ab301-3b64-4994-9379-711e8a6d4317 zone_kind = external_dns
60702025-12-23T22:03:04.692ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = SledList step id = Fetch
60712025-12-23T22:03:04.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60722025-12-23T22:03:04.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
60732025-12-23T22:03:04.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60742025-12-23T22:03:04.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60752025-12-23T22:03:04.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60762025-12-23T22:03:04.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60772025-12-23T22:03:04.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60782025-12-23T22:03:04.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60792025-12-23T22:03:04.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
60802025-12-23T22:03:04.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60812025-12-23T22:03:04.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60822025-12-23T22:03:04.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60832025-12-23T22:03:04.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60842025-12-23T22:03:04.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60852025-12-23T22:03:04.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60862025-12-23T22:03:04.709ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
60872025-12-23T22:03:04.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60882025-12-23T22:03:04.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60892025-12-23T22:03:04.713ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = DeployNexusRecords step id = Ensure
60902025-12-23T22:03:04.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60912025-12-23T22:03:04.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60922025-12-23T22:03:04.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
60932025-12-23T22:03:04.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60942025-12-23T22:03:04.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60952025-12-23T22:03:04.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60962025-12-23T22:03:04.719ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
60972025-12-23T22:03:04.719ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
60982025-12-23T22:03:04.719ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
60992025-12-23T22:03:04.719ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61002025-12-23T22:03:04.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61012025-12-23T22:03:04.722ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
61022025-12-23T22:03:04.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61032025-12-23T22:03:04.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61042025-12-23T22:03:04.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61052025-12-23T22:03:04.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61062025-12-23T22:03:04.723ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61072025-12-23T22:03:04.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61082025-12-23T22:03:04.729ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61092025-12-23T22:03:04.729ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61102025-12-23T22:03:04.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61112025-12-23T22:03:04.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61122025-12-23T22:03:04.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61132025-12-23T22:03:04.731ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61142025-12-23T22:03:04.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
61152025-12-23T22:03:04.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61162025-12-23T22:03:04.733ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
61172025-12-23T22:03:04.733ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
61182025-12-23T22:03:04.733ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61192025-12-23T22:03:04.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61202025-12-23T22:03:04.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
61212025-12-23T22:03:04.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61222025-12-23T22:03:04.735ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61232025-12-23T22:03:04.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61242025-12-23T22:03:04.736ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61252025-12-23T22:03:04.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61262025-12-23T22:03:04.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
61272025-12-23T22:03:04.741ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61282025-12-23T22:03:04.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
61292025-12-23T22:03:04.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
61302025-12-23T22:03:04.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
61312025-12-23T22:03:04.744ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61322025-12-23T22:03:04.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61332025-12-23T22:03:04.745ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
61342025-12-23T22:03:04.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61352025-12-23T22:03:04.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61362025-12-23T22:03:04.749ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
61372025-12-23T22:03:04.749ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
61382025-12-23T22:03:04.749ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-23 21:50:07.200941 UTC dns_group = external generation = 2
61392025-12-23T22:03:04.749ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 129.825564ms iteration = 14 status = {"generation":2}
61402025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
61412025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
61422025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
61432025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
61442025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
61452025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
61462025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
61472025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
61482025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61492025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
61502025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
61512025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61522025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61532025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
61542025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.44\u{b5}s iteration = 27 status = {"disabled":true,"errors":[],"sps":[]}
61552025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
61562025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 40 reason = Timeout
61572025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
61582025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 53 reason = Timeout
61592025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
61602025-12-23T22:03:04.755ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
61612025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
61622025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
61632025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 53 reason = Timeout
61642025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
61652025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 69 reason = Timeout
61662025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
61672025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 157 reason = Timeout
61682025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61692025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
61702025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
61712025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 43 reason = Timeout
61722025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61732025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
61742025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61752025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 40 reason = Timeout
61762025-12-23T22:03:04.756ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
61772025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61782025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
61792025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
61802025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
61812025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
61822025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61832025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
61842025-12-23T22:03:04.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
61852025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
61862025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
61872025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
61882025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
61892025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
61902025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
61912025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
61922025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61932025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61942025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
61952025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
61962025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
61972025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
61982025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61992025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
62002025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62012025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
62022025-12-23T22:03:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
62032025-12-23T22:03:04.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
62042025-12-23T22:03:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62052025-12-23T22:03:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
62062025-12-23T22:03:04.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62072025-12-23T22:03:04.761ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62082025-12-23T22:03:04.763ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62092025-12-23T22:03:04.763ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 7.09968ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
62102025-12-23T22:03:04.763ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62112025-12-23T22:03:04.763ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62122025-12-23T22:03:04.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62132025-12-23T22:03:04.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62142025-12-23T22:03:04.765ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62152025-12-23T22:03:04.765ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62162025-12-23T22:03:04.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62172025-12-23T22:03:04.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62182025-12-23T22:03:04.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62192025-12-23T22:03:04.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
62202025-12-23T22:03:04.767ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
62212025-12-23T22:03:04.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
62222025-12-23T22:03:04.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62232025-12-23T22:03:04.768ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62242025-12-23T22:03:04.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62252025-12-23T22:03:04.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62262025-12-23T22:03:04.769ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62272025-12-23T22:03:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
62282025-12-23T22:03:04.769ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
62292025-12-23T22:03:04.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62302025-12-23T22:03:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
62312025-12-23T22:03:04.770ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
62322025-12-23T22:03:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
62332025-12-23T22:03:04.770ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.930422ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62342025-12-23T22:03:04.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
62352025-12-23T22:03:04.771ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62362025-12-23T22:03:04.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62372025-12-23T22:03:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
62382025-12-23T22:03:04.772ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62392025-12-23T22:03:04.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62402025-12-23T22:03:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
62412025-12-23T22:03:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62422025-12-23T22:03:04.772ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 16.938178ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
62432025-12-23T22:03:04.773ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
62442025-12-23T22:03:04.773ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 17.880675ms iteration = 157 status = {"config_updated":false}
62452025-12-23T22:03:04.774ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
62462025-12-23T22:03:04.774ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
62472025-12-23T22:03:04.775ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62482025-12-23T22:03:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
62492025-12-23T22:03:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62502025-12-23T22:03:04.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62512025-12-23T22:03:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
62522025-12-23T22:03:04.776ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 388fa4f3-ccd9-44a0-8b06-fc205989c366 original_time_started = 2025-12-23 22:02:06.170182 UTC
62532025-12-23T22:03:04.776ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 20.327394ms iteration = 69 status = {"Loaded":{"collection_id":"388fa4f3-ccd9-44a0-8b06-fc205989c366","time_loaded":"2025-12-23T22:03:04.755991812Z","time_started":"2025-12-23T22:02:06.170182Z"}}
62542025-12-23T22:03:04.776ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 876080756 } })
62552025-12-23T22:03:04.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
62562025-12-23T22:03:04.777ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62572025-12-23T22:03:04.777ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
62582025-12-23T22:03:04.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
62592025-12-23T22:03:04.777ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 877118815 } })
62602025-12-23T22:03:04.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
62612025-12-23T22:03:04.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62622025-12-23T22:03:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62632025-12-23T22:03:04.779ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
62642025-12-23T22:03:04.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62652025-12-23T22:03:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
62662025-12-23T22:03:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
62672025-12-23T22:03:04.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62682025-12-23T22:03:04.780ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62692025-12-23T22:03:04.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62702025-12-23T22:03:04.780ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
62712025-12-23T22:03:04.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
62722025-12-23T22:03:04.781ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.278774ms iteration = 27 status = 0
62732025-12-23T22:03:04.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62742025-12-23T22:03:04.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62752025-12-23T22:03:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62762025-12-23T22:03:04.790ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62772025-12-23T22:03:04.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62782025-12-23T22:03:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
62792025-12-23T22:03:04.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62802025-12-23T22:03:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
62812025-12-23T22:03:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
62822025-12-23T22:03:04.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
62832025-12-23T22:03:04.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62842025-12-23T22:03:04.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62852025-12-23T22:03:04.796ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
62862025-12-23T22:03:04.796ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
62872025-12-23T22:03:04.796ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 40.681089ms iteration = 53 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
62882025-12-23T22:03:04.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62892025-12-23T22:03:04.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62902025-12-23T22:03:04.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62912025-12-23T22:03:04.805ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
62922025-12-23T22:03:04.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
62932025-12-23T22:03:04.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62942025-12-23T22:03:04.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62952025-12-23T22:03:04.826ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
62962025-12-23T22:03:04.826ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.755747ms iteration = 79 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
62972025-12-23T22:03:04.838ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
62982025-12-23T22:03:04.838ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
62992025-12-23T22:03:04.839ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef5e79fc-c8c6-488a-9289-ce031cb06081", "content-length": "26", "date": "Tue, 23 Dec 2025 22:03:04 GMT"} })
63002025-12-23T22:03:04.839ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63012025-12-23T22:03:04.839ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.429075ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
63022025-12-23T22:03:04.839ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63032025-12-23T22:03:04.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63042025-12-23T22:03:04.842ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
63052025-12-23T22:03:04.843ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1682, tv_nsec: 942936213 } })
63062025-12-23T22:03:04.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63072025-12-23T22:03:04.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63082025-12-23T22:03:04.848ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63092025-12-23T22:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63102025-12-23T22:03:04.849ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63112025-12-23T22:03:04.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63122025-12-23T22:03:04.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63132025-12-23T22:03:04.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
63142025-12-23T22:03:04.854ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
63152025-12-23T22:03:04.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
63162025-12-23T22:03:04.854ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63172025-12-23T22:03:04.854ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63182025-12-23T22:03:04.854ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
63192025-12-23T22:03:04.854ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63202025-12-23T22:03:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
63212025-12-23T22:03:04.860ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = SledAgent step id = Ensure
63222025-12-23T22:03:04.860ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-23T21:50:00.171530Z, time_modified: 2025-12-23T21:50:00.171530Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 2a7ec183-a561-48a8-be39-417a54aa2c01, db_silo: Silo { identity: SiloIdentity { id: 2a7ec183-a561-48a8-be39-417a54aa2c01, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-23T21:50:07.128853Z, time_modified: 2025-12-23T21:50:07.128853Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e" }] }) }
63232025-12-23T22:03:04.860ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 238.113151ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]}},"default_endpoint":{"silo_id":"2a7ec183-a561-48a8-be39-417a54aa2c01","tls_certs":["8e89eeb39bac4bdc33dd32bf8993699fd5b5fa6fddba796f6d326a5475861d7e"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
63242025-12-23T22:03:04.860ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63252025-12-23T22:03:04.860ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
63262025-12-23T22:03:04.860ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63272025-12-23T22:03:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
63282025-12-23T22:03:04.862ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 106.249339ms iteration = 53 status = "NoSitrep"
63292025-12-23T22:03:04.862ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63302025-12-23T22:03:04.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63312025-12-23T22:03:04.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63322025-12-23T22:03:04.863ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 107.632518ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
63332025-12-23T22:03:04.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
63342025-12-23T22:03:04.864ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63352025-12-23T22:03:04.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
63362025-12-23T22:03:04.865ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63372025-12-23T22:03:04.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63382025-12-23T22:03:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63392025-12-23T22:03:04.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63402025-12-23T22:03:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63412025-12-23T22:03:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
63422025-12-23T22:03:04.871ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63432025-12-23T22:03:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63442025-12-23T22:03:04.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63452025-12-23T22:03:04.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63462025-12-23T22:03:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63472025-12-23T22:03:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63482025-12-23T22:03:04.873ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
63492025-12-23T22:03:04.873ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 118.005781ms iteration = 40 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
63502025-12-23T22:03:04.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63512025-12-23T22:03:04.878ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63522025-12-23T22:03:04.878ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63532025-12-23T22:03:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63542025-12-23T22:03:04.878ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63552025-12-23T22:03:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63562025-12-23T22:03:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63572025-12-23T22:03:04.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63582025-12-23T22:03:04.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63592025-12-23T22:03:04.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63602025-12-23T22:03:04.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63612025-12-23T22:03:04.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63622025-12-23T22:03:04.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63632025-12-23T22:03:04.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63642025-12-23T22:03:04.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63652025-12-23T22:03:04.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63662025-12-23T22:03:04.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63672025-12-23T22:03:04.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63682025-12-23T22:03:04.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63692025-12-23T22:03:04.914ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
63702025-12-23T22:03:04.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63712025-12-23T22:03:04.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63722025-12-23T22:03:04.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63732025-12-23T22:03:04.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63742025-12-23T22:03:04.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63752025-12-23T22:03:04.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63762025-12-23T22:03:04.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 165.949325ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
63772025-12-23T22:03:04.926ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
63782025-12-23T22:03:04.928ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63792025-12-23T22:03:04.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63802025-12-23T22:03:04.935ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
63812025-12-23T22:03:04.938ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 318.381277ms iteration = 27 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
63822025-12-23T22:03:04.944ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63832025-12-23T22:03:04.944ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
63842025-12-23T22:03:04.944ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 188.437229ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
63852025-12-23T22:03:04.946ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
63862025-12-23T22:03:04.947ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "f3393314-4f01-441e-91d0-8b8e51012be7", "date": "Tue, 23 Dec 2025 22:03:04 GMT"} })
63872025-12-23T22:03:04.947ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
63882025-12-23T22:03:04.949ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
63892025-12-23T22:03:04.949ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 193.137922ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
63902025-12-23T22:03:04.950ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
63912025-12-23T22:03:04.950ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 307.164161ms iteration = 14 status = {"errors":[],"probes_by_sled":{"ff85f534-787c-483e-ba89-a37cdea1dd38":0}}
63922025-12-23T22:03:04.958ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
63932025-12-23T22:03:04.983ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
63942025-12-23T22:03:04.983ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
63952025-12-23T22:03:04.983ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "617f89b5-8142-4327-b177-1f470f4c3a8d", "content-length": "26", "date": "Tue, 23 Dec 2025 22:03:04 GMT"} })
63962025-12-23T22:03:04.983ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
63972025-12-23T22:03:04.986ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
63982025-12-23T22:03:04.992ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
63992025-12-23T22:03:04.992ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 349.165816ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
64002025-12-23T22:03:04.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64012025-12-23T22:03:04.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64022025-12-23T22:03:04.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64032025-12-23T22:03:04.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64042025-12-23T22:03:04.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64052025-12-23T22:03:05.003ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
64062025-12-23T22:03:05.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64072025-12-23T22:03:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64082025-12-23T22:03:05.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64092025-12-23T22:03:05.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64102025-12-23T22:03:05.010ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 253.99583ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
64112025-12-23T22:03:05.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64122025-12-23T22:03:05.014ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
64132025-12-23T22:03:05.015ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 259.32096ms iteration = 43 status = {"physical_disks_added":0}
64142025-12-23T22:03:05.015ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "cfc5ed18-09c0-41a7-a363-0e326bc7d3b5", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64152025-12-23T22:03:05.015ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64162025-12-23T22:03:05.015ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64172025-12-23T22:03:05.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
64182025-12-23T22:03:05.016ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
64192025-12-23T22:03:05.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64202025-12-23T22:03:05.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64212025-12-23T22:03:05.018ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64222025-12-23T22:03:05.018ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "adc63b3f-4fd0-4fe5-b6b5-5ccc5826acba", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64232025-12-23T22:03:05.018ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64242025-12-23T22:03:05.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64252025-12-23T22:03:05.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64262025-12-23T22:03:05.021ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
64272025-12-23T22:03:05.021ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 401.835472ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
64282025-12-23T22:03:05.022ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64292025-12-23T22:03:05.022ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64302025-12-23T22:03:05.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
64312025-12-23T22:03:05.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64322025-12-23T22:03:05.024ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64332025-12-23T22:03:05.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
64342025-12-23T22:03:05.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64352025-12-23T22:03:05.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64362025-12-23T22:03:05.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64372025-12-23T22:03:05.027ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64382025-12-23T22:03:05.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
64392025-12-23T22:03:05.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64402025-12-23T22:03:05.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64412025-12-23T22:03:05.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64422025-12-23T22:03:05.035ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
64432025-12-23T22:03:05.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64442025-12-23T22:03:05.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64452025-12-23T22:03:05.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64462025-12-23T22:03:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64472025-12-23T22:03:05.053ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 433.180087ms iteration = 27 status = {}
64482025-12-23T22:03:05.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64492025-12-23T22:03:05.057ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
64502025-12-23T22:03:05.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64512025-12-23T22:03:05.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64522025-12-23T22:03:05.069ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
64532025-12-23T22:03:05.071ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "333d796f-fb63-48d9-adb0-55a60c68a3bd", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} }) sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
64542025-12-23T22:03:05.073ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = FirewallRules step id = Ensure
64552025-12-23T22:03:05.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64562025-12-23T22:03:05.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64572025-12-23T22:03:05.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64582025-12-23T22:03:05.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64592025-12-23T22:03:05.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64602025-12-23T22:03:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64612025-12-23T22:03:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64622025-12-23T22:03:05.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64632025-12-23T22:03:05.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64642025-12-23T22:03:05.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64652025-12-23T22:03:05.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64662025-12-23T22:03:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64672025-12-23T22:03:05.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64682025-12-23T22:03:05.089ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
64692025-12-23T22:03:05.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
64702025-12-23T22:03:05.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64712025-12-23T22:03:05.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64722025-12-23T22:03:05.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64732025-12-23T22:03:05.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64742025-12-23T22:03:05.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64752025-12-23T22:03:05.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64762025-12-23T22:03:05.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64772025-12-23T22:03:05.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
64782025-12-23T22:03:05.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64792025-12-23T22:03:05.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64802025-12-23T22:03:05.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64812025-12-23T22:03:05.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64822025-12-23T22:03:05.102ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
64832025-12-23T22:03:05.102ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
64842025-12-23T22:03:05.103ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8352e421-6689-4eb9-9a9e-5a272eda45d8 parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
64852025-12-23T22:03:05.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64862025-12-23T22:03:05.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64872025-12-23T22:03:05.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64882025-12-23T22:03:05.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64892025-12-23T22:03:05.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64902025-12-23T22:03:05.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64912025-12-23T22:03:05.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64922025-12-23T22:03:05.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64932025-12-23T22:03:05.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64942025-12-23T22:03:05.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64952025-12-23T22:03:05.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
64962025-12-23T22:03:05.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64972025-12-23T22:03:05.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64982025-12-23T22:03:05.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64992025-12-23T22:03:05.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65002025-12-23T22:03:05.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65012025-12-23T22:03:05.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65022025-12-23T22:03:05.113ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
65032025-12-23T22:03:05.113ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
65042025-12-23T22:03:05.114ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 473.132765ms iteration = 32 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
65052025-12-23T22:03:05.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
65062025-12-23T22:03:05.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65072025-12-23T22:03:05.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65082025-12-23T22:03:05.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65092025-12-23T22:03:05.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65102025-12-23T22:03:05.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65112025-12-23T22:03:05.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65122025-12-23T22:03:05.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65132025-12-23T22:03:05.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65142025-12-23T22:03:05.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65152025-12-23T22:03:05.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65162025-12-23T22:03:05.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65172025-12-23T22:03:05.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65182025-12-23T22:03:05.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65192025-12-23T22:03:05.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65202025-12-23T22:03:05.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65212025-12-23T22:03:05.135ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1683, tv_nsec: 234625697 } })
65222025-12-23T22:03:05.139ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
65232025-12-23T22:03:05.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65242025-12-23T22:03:05.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65252025-12-23T22:03:05.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65262025-12-23T22:03:05.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65272025-12-23T22:03:05.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65282025-12-23T22:03:05.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65292025-12-23T22:03:05.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65302025-12-23T22:03:05.150ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
65312025-12-23T22:03:05.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65322025-12-23T22:03:05.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65332025-12-23T22:03:05.153ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
65342025-12-23T22:03:05.153ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65352025-12-23T22:03:05.154ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9f2c620-7cf0-4dfe-a53b-a407c5b83d24", "content-length": "372", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
65362025-12-23T22:03:05.154ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65372025-12-23T22:03:05.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65382025-12-23T22:03:05.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65392025-12-23T22:03:05.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65402025-12-23T22:03:05.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65412025-12-23T22:03:05.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65422025-12-23T22:03:05.159ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
65432025-12-23T22:03:05.160ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "902fcc11-951a-495e-884b-3edf272873ea", "content-length": "691", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
65442025-12-23T22:03:05.160ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
65452025-12-23T22:03:05.160ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
65462025-12-23T22:03:05.160ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
65472025-12-23T22:03:05.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65482025-12-23T22:03:05.162ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 406.173326ms iteration = 27 status = {}
65492025-12-23T22:03:05.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65502025-12-23T22:03:05.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65512025-12-23T22:03:05.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65522025-12-23T22:03:05.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65532025-12-23T22:03:05.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65542025-12-23T22:03:05.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65552025-12-23T22:03:05.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65562025-12-23T22:03:05.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65572025-12-23T22:03:05.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65582025-12-23T22:03:05.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65592025-12-23T22:03:05.179ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
65602025-12-23T22:03:05.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65612025-12-23T22:03:05.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65622025-12-23T22:03:05.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65632025-12-23T22:03:05.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65642025-12-23T22:03:05.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65652025-12-23T22:03:05.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65662025-12-23T22:03:05.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65672025-12-23T22:03:05.185ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
65682025-12-23T22:03:05.186ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "cd01dcf5-51c4-4501-9a63-241f0bf6f9e8", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65692025-12-23T22:03:05.186ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65702025-12-23T22:03:05.188ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "09e650a3-c057-4837-881b-6a584bf19f47", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65712025-12-23T22:03:05.188ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65722025-12-23T22:03:05.188ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
65732025-12-23T22:03:05.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65742025-12-23T22:03:05.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65752025-12-23T22:03:05.190ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "6894e8d9-c2c7-4ddd-8d37-8d08db97b8a8", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65762025-12-23T22:03:05.190ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65772025-12-23T22:03:05.193ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
65782025-12-23T22:03:05.193ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 573.521459ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
65792025-12-23T22:03:05.195ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
65802025-12-23T22:03:05.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65812025-12-23T22:03:05.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65822025-12-23T22:03:05.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65832025-12-23T22:03:05.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
65842025-12-23T22:03:05.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65852025-12-23T22:03:05.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65862025-12-23T22:03:05.201ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
65872025-12-23T22:03:05.201ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1683, tv_nsec: 301412423 } })
65882025-12-23T22:03:05.201ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 04a2d604-085f-4cb0-a089-0acb5a8731c9, kind: Service { id: e34ab301-3b64-4994-9379-711e8a6d4317 }, name: Name("external-dns-e34ab301-3b64-4994-9379-711e8a6d4317"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 133, 23])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6e38bc74-6814-49f9-b2ff-25e22b33c858, kind: Service { id: 2bc6e36d-0fab-4746-9740-073e15922b00 }, name: Name("external-dns-2bc6e36d-0fab-4746-9740-073e15922b00"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 193])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 3dab0b67-7237-438d-9073-2c211572270d, kind: Service { id: 570ca1d2-2fc4-4ef8-9a95-86cebdab47dd }, name: Name("nexus-570ca1d2-2fc4-4ef8-9a95-86cebdab47dd"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 215, 193])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 5696403f-7399-48fa-a5ac-95e68bc4f2a9, kind: Service { id: a981c0ae-aa35-4460-b263-22312db6f302 }, name: Name("nexus-a981c0ae-aa35-4460-b263-22312db6f302"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 203, 18])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9fd07ebb-0432-440a-907b-076e4886e3f1, kind: Service { id: 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21 }, name: Name("nexus-8e593c8b-acdb-44d0-8ac2-98e0fc1adb21"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 63])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
65892025-12-23T22:03:05.201ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
65902025-12-23T22:03:05.219ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
65912025-12-23T22:03:05.219ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
65922025-12-23T22:03:05.219ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "374675be-ac43-4b42-8414-5b06591fd1ae", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
65932025-12-23T22:03:05.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65942025-12-23T22:03:05.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65952025-12-23T22:03:05.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65962025-12-23T22:03:05.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65972025-12-23T22:03:05.225ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65982025-12-23T22:03:05.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65992025-12-23T22:03:05.228ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66002025-12-23T22:03:05.228ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66012025-12-23T22:03:05.229ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a41e8de8-e6ff-4661-bc6a-bab12c13d296", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66022025-12-23T22:03:05.231ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 475.374819ms iteration = 40 status = {}
66032025-12-23T22:03:05.231ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 41 reason = Dependency
66042025-12-23T22:03:05.231ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
66052025-12-23T22:03:05.231ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66062025-12-23T22:03:05.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66072025-12-23T22:03:05.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66082025-12-23T22:03:05.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66092025-12-23T22:03:05.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66102025-12-23T22:03:05.235ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
66112025-12-23T22:03:05.235ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66122025-12-23T22:03:05.236ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
66132025-12-23T22:03:05.236ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66142025-12-23T22:03:05.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66152025-12-23T22:03:05.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66162025-12-23T22:03:05.237ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66172025-12-23T22:03:05.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
66182025-12-23T22:03:05.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66192025-12-23T22:03:05.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66202025-12-23T22:03:05.248ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
66212025-12-23T22:03:05.248ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
66222025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
66232025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
66242025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
66252025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
66262025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
66272025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
66282025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
66292025-12-23T22:03:05.249ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
66302025-12-23T22:03:05.250ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
66312025-12-23T22:03:05.250ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66322025-12-23T22:03:05.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66332025-12-23T22:03:05.254ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
66342025-12-23T22:03:05.254ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
66352025-12-23T22:03:05.277ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
66362025-12-23T22:03:05.278ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66372025-12-23T22:03:05.282ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "09dbf5b5-8706-4e15-8a8c-8ea26bce7e2b", "content-length": "372", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66382025-12-23T22:03:05.282ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66392025-12-23T22:03:05.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66402025-12-23T22:03:05.294ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
66412025-12-23T22:03:05.294ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
66422025-12-23T22:03:05.300ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b82b209d-ab56-4f03-95f2-850304bcf516", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66432025-12-23T22:03:05.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66442025-12-23T22:03:05.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66452025-12-23T22:03:05.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66462025-12-23T22:03:05.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66472025-12-23T22:03:05.306ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66482025-12-23T22:03:05.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66492025-12-23T22:03:05.309ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66502025-12-23T22:03:05.309ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66512025-12-23T22:03:05.310ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ded9f08a-56be-4f89-840b-9002bb351833", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66522025-12-23T22:03:05.313ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 82.483908ms iteration = 41 status = {}
66532025-12-23T22:03:05.350ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), time_created: 2025-12-23T21:49:57.392343Z, time_modified: 2025-12-23T21:49:57.392343Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
66542025-12-23T22:03:05.350ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
66552025-12-23T22:03:05.388ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f601193e-a9f3-4314-a243-b361a5ab539b", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66562025-12-23T22:03:05.388ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
66572025-12-23T22:03:05.388ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
66582025-12-23T22:03:05.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66592025-12-23T22:03:05.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66602025-12-23T22:03:05.394ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "770da787-f598-4de1-9c66-f456c93f5ac7", "content-length": "682", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66612025-12-23T22:03:05.394ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66622025-12-23T22:03:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66632025-12-23T22:03:05.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66642025-12-23T22:03:05.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66652025-12-23T22:03:05.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66662025-12-23T22:03:05.398ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66672025-12-23T22:03:05.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66682025-12-23T22:03:05.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
66692025-12-23T22:03:05.401ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
66702025-12-23T22:03:05.402ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66712025-12-23T22:03:05.402ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "1fbad183-c369-4910-bc1b-98d8bc08dc8f", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
66722025-12-23T22:03:05.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66732025-12-23T22:03:05.403ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
66742025-12-23T22:03:05.403ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = Dns step id = Ensure
66752025-12-23T22:03:05.403ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
66762025-12-23T22:03:05.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66772025-12-23T22:03:05.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66782025-12-23T22:03:05.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66792025-12-23T22:03:05.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66802025-12-23T22:03:05.405ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
66812025-12-23T22:03:05.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
66822025-12-23T22:03:05.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66832025-12-23T22:03:05.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
66842025-12-23T22:03:05.411ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
66852025-12-23T22:03:05.411ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
66862025-12-23T22:03:05.411ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
66872025-12-23T22:03:05.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66882025-12-23T22:03:05.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66892025-12-23T22:03:05.411ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 655.673448ms iteration = 27 status = {}
66902025-12-23T22:03:05.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66912025-12-23T22:03:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
66922025-12-23T22:03:05.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66932025-12-23T22:03:05.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66942025-12-23T22:03:05.466ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
66952025-12-23T22:03:05.466ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
66962025-12-23T22:03:05.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66972025-12-23T22:03:05.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66982025-12-23T22:03:05.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66992025-12-23T22:03:05.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
67002025-12-23T22:03:05.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67012025-12-23T22:03:05.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67022025-12-23T22:03:05.474ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = 30d0e501-f534-4b3a-b120-b36556dea8e9 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
67032025-12-23T22:03:05.474ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
67042025-12-23T22:03:05.474ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
67052025-12-23T22:03:05.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67062025-12-23T22:03:05.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67072025-12-23T22:03:05.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67082025-12-23T22:03:05.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67092025-12-23T22:03:05.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67102025-12-23T22:03:05.480ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
67112025-12-23T22:03:05.481ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
67122025-12-23T22:03:05.481ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
67132025-12-23T22:03:05.481ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
67142025-12-23T22:03:05.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67152025-12-23T22:03:05.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67162025-12-23T22:03:05.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67172025-12-23T22:03:05.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
67182025-12-23T22:03:05.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67192025-12-23T22:03:05.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67202025-12-23T22:03:05.492ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
67212025-12-23T22:03:05.492ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 0 version = 2
67222025-12-23T22:03:05.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67232025-12-23T22:03:05.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67242025-12-23T22:03:05.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67252025-12-23T22:03:05.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
67262025-12-23T22:03:05.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67272025-12-23T22:03:05.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67282025-12-23T22:03:05.500ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 83bf91e4-ea4d-4bc8-8212-582297f992d1 dns_zone_name = oxide.test found_so_far = 4 version = 2
67292025-12-23T22:03:05.500ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
67302025-12-23T22:03:05.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67312025-12-23T22:03:05.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67322025-12-23T22:03:05.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67332025-12-23T22:03:05.504ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
67342025-12-23T22:03:05.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67352025-12-23T22:03:05.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67362025-12-23T22:03:05.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67372025-12-23T22:03:05.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67382025-12-23T22:03:05.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67392025-12-23T22:03:05.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67402025-12-23T22:03:05.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67412025-12-23T22:03:05.512ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
67422025-12-23T22:03:05.514ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
67432025-12-23T22:03:05.514ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
67442025-12-23T22:03:05.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = OmicronZones step id = Cleanup
67452025-12-23T22:03:05.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = OmicronZones step id = Cleanup
67462025-12-23T22:03:05.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = PhysicalDisks step id = Cleanup
67472025-12-23T22:03:05.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = Clickhouse step id = Ensure
67482025-12-23T22:03:05.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = Clickhouse step id = Ensure
67492025-12-23T22:03:05.545ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
67502025-12-23T22:03:05.550ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "54760928-79a2-43f7-9c2f-361148390c4a", "date": "Tue, 23 Dec 2025 22:03:05 GMT"} })
67512025-12-23T22:03:05.551ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = SupportBundles step id = Cleanup
67522025-12-23T22:03:05.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67532025-12-23T22:03:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67542025-12-23T22:03:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67552025-12-23T22:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
67562025-12-23T22:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67572025-12-23T22:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67582025-12-23T22:03:05.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67592025-12-23T22:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67602025-12-23T22:03:05.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67612025-12-23T22:03:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
67622025-12-23T22:03:05.602ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = OmicronZones step id = Cleanup
67632025-12-23T22:03:05.602ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
67642025-12-23T22:03:05.602ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
67652025-12-23T22:03:05.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67662025-12-23T22:03:05.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67672025-12-23T22:03:05.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67682025-12-23T22:03:05.606ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
67692025-12-23T22:03:05.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67702025-12-23T22:03:05.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67712025-12-23T22:03:05.654ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
67722025-12-23T22:03:05.654ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(02039deb-aee8-405c-80d5-e584bd199fe1 (blueprint), false)
67732025-12-23T22:03:05.654ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = Cockroach step id = Ensure
67742025-12-23T22:03:05.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67752025-12-23T22:03:05.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67762025-12-23T22:03:05.687ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
67772025-12-23T22:03:05.687ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2a2b5393-aed1-4af0-b505-913d37c41c58 step component = MgsUpdates step id = Ensure
67782025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 41 reason = Dependency
67792025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 17 reason = Dependency
67802025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
67812025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67822025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.067720629s iteration = 15 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3190,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":73036401,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":73036352,"secs":0}},"event_index":1,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":73049272,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":20777647,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":20777596,"secs":0}},"event_index":2,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":93841408,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":143229773,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":143229723,"secs":0}},"event_index":3,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":237098762,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":216497531,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":216497501,"secs":0}},"event_index":4,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":453610624,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":330394697,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":330394667,"secs":0}},"event_index":5,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":784016401,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":110615713,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":110615673,"secs":0}},"event_index":6,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":894639794,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6170,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":6130,"secs":0}},"event_index":7,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":894659074,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2600,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2560,"secs":0}},"event_index":8,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":894664344,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3169,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3129,"secs":0}},"event_index":9,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":894669034,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1680,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1640,"secs":0}},"event_index":10,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":894672425,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36684295,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36684255,"secs":0}},"event_index":11,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":931357790,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51321128,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":51321088,"secs":0}},"event_index":12,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":982688809,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51990367,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":51990337,"secs":0}},"event_index":13,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":34696207,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":32645952,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":32645912,"secs":0}},"event_index":14,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":67350249,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4980,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4950,"secs":0}},"event_index":15,"execution_id":"a64914d1-a1b2-4d07-a096-d6d842f88b05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":67363489,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"02039deb-aee8-405c-80d5-e584bd199fe1"}
67832025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
67842025-12-23T22:03:05.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67852025-12-23T22:03:05.689ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67862025-12-23T22:03:05.689ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67872025-12-23T22:03:05.692ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
67882025-12-23T22:03:05.692ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
67892025-12-23T22:03:05.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
67902025-12-23T22:03:05.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
67912025-12-23T22:03:05.692ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67922025-12-23T22:03:05.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67932025-12-23T22:03:05.694ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67942025-12-23T22:03:05.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67952025-12-23T22:03:05.695ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67962025-12-23T22:03:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
67972025-12-23T22:03:05.698ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67982025-12-23T22:03:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67992025-12-23T22:03:05.699ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
68002025-12-23T22:03:05.699ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.624926ms iteration = 41 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
68012025-12-23T22:03:05.750ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(c52dbd67-418f-4c46-9f2c-a5d9fa4bf644, 0), (7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e, 0), (9403537f-2397-43e4-8842-eb9c54c7d9fa, 0), (7dfe3137-2631-424d-b6f9-782ed4f98966, 0)] collection_id = c52dbd67-418f-4c46-9f2c-a5d9fa4bf644
68022025-12-23T22:03:05.750ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68032025-12-23T22:03:05.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68042025-12-23T22:03:05.864ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = c52dbd67-418f-4c46-9f2c-a5d9fa4bf644 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
68052025-12-23T22:03:05.864ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68062025-12-23T22:03:05.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68072025-12-23T22:03:05.869ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e, 0), (9403537f-2397-43e4-8842-eb9c54c7d9fa, 0), (7dfe3137-2631-424d-b6f9-782ed4f98966, 0), (388fa4f3-ccd9-44a0-8b06-fc205989c366, 0)] collection_id = 7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e
68082025-12-23T22:03:05.869ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68092025-12-23T22:03:05.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68102025-12-23T22:03:05.973ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 7aed0cbe-45fa-432b-aeb1-5b7dd3dbb66e file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
68112025-12-23T22:03:05.973ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68122025-12-23T22:03:05.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68132025-12-23T22:03:06.013ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(9403537f-2397-43e4-8842-eb9c54c7d9fa, 0), (7dfe3137-2631-424d-b6f9-782ed4f98966, 0), (388fa4f3-ccd9-44a0-8b06-fc205989c366, 0)]
68142025-12-23T22:03:06.013ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1684, tv_nsec: 113404764 } })
68152025-12-23T22:03:06.045ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("01b204c0-22af-4353-ad94-4c2c0d1eb01b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70f89219-475c-48c6-a8c8-388723cccdcf.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("7b633ce6-0ad9-42aa-8164-bc5c5f301b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("eaafa87a-91ad-4109-aa73-7065c531ed97.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("fad25b51-dcca-4686-9581-11ff0c646670.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1684, tv_nsec: 145517520 } })
68162025-12-23T22:03:06.046ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
68172025-12-23T22:03:06.046ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
68182025-12-23T22:03:06.046ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
68192025-12-23T22:03:06.047ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "85c4d5e5-dcf7-4425-bae2-5772a983bc5d", "content-length": "476", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68202025-12-23T22:03:06.047ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
68212025-12-23T22:03:06.047ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "424c7c8b-e6a3-4363-87cc-4d7d77683380", "content-length": "734", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68222025-12-23T22:03:06.047ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
68232025-12-23T22:03:06.048ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09ff7025-a244-47c9-9ea5-59a99e6cc415", "content-length": "111", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68242025-12-23T22:03:06.048ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
68252025-12-23T22:03:06.048ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55dd1ada-ea3f-4175-ac14-540f1909715c", "content-length": "111", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68262025-12-23T22:03:06.048ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
68272025-12-23T22:03:06.049ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92b37819-720b-4544-bc7c-005788905b3c", "content-length": "163", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68282025-12-23T22:03:06.049ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
68292025-12-23T22:03:06.049ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92bcd352-7c5b-41cf-a73e-7f05e8af410c", "content-length": "163", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68302025-12-23T22:03:06.049ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
68312025-12-23T22:03:06.050ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccf32fdc-829c-44c5-ae96-00e0ee8e272a", "content-length": "166", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68322025-12-23T22:03:06.050ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
68332025-12-23T22:03:06.050ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6310ee66-ccd5-46f5-a022-f0f83e3b38ec", "content-length": "166", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68342025-12-23T22:03:06.050ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
68352025-12-23T22:03:06.050ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e7aa6c6-c4f4-44d4-93bb-5a2ab2cda3bf", "content-length": "702", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68362025-12-23T22:03:06.050ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
68372025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34f89436-841f-4d46-b8aa-3650bfff0a36", "content-length": "727", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68382025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
68392025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce06fd17-02cf-40fe-9724-11fb74b80860", "content-length": "729", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68402025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
68412025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a76c909e-308d-4df9-9059-efbe2f10599f", "content-length": "728", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68422025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
68432025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38a7482d-ddac-49a7-b83d-ec31eb651e27", "content-length": "722", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68442025-12-23T22:03:06.051ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
68452025-12-23T22:03:06.052ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bbe2ad3-b633-4151-a578-5ebb8ee58053", "content-length": "10", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68462025-12-23T22:03:06.052ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
68472025-12-23T22:03:06.052ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c79ec95-600d-4ec2-8ef1-65513cf0d2ae", "content-length": "144", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68482025-12-23T22:03:06.052ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
68492025-12-23T22:03:06.052ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72639d20-7f94-4686-b675-5458d4ee0bdb", "content-length": "147", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68502025-12-23T22:03:06.052ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
68512025-12-23T22:03:06.053ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccabfd59-928a-45d6-ba85-1b5df9cab98a", "content-length": "109", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68522025-12-23T22:03:06.053ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
68532025-12-23T22:03:06.053ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "558d6177-8539-412b-9803-72912d6e5a2c", "content-length": "109", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68542025-12-23T22:03:06.053ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
68552025-12-23T22:03:06.054ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3dc8779-7323-4c4b-9ffd-721fa8a323cf", "content-length": "163", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68562025-12-23T22:03:06.054ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
68572025-12-23T22:03:06.054ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccbd7ee2-9246-44f6-acad-95565d3ecbef", "content-length": "163", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68582025-12-23T22:03:06.054ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
68592025-12-23T22:03:06.055ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49b92083-88da-47fd-bf10-efd89c99ea94", "content-length": "166", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68602025-12-23T22:03:06.055ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
68612025-12-23T22:03:06.055ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e4c70b8-0e70-468f-83a7-3fc7f8f9ab2c", "content-length": "166", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68622025-12-23T22:03:06.055ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
68632025-12-23T22:03:06.055ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5a792b4-16b9-4d08-9362-9946853d0d69", "content-length": "702", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68642025-12-23T22:03:06.055ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
68652025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e500e9a2-1bfb-41ee-bd6b-54ea1edbf1fa", "content-length": "727", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68662025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
68672025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95980c1a-937e-4cb5-a3ee-a43961af5d59", "content-length": "729", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68682025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
68692025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "451c562a-7e41-4c62-a782-416a6b3b353e", "content-length": "728", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68702025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68712025-12-23T22:03:06.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68722025-12-23T22:03:06.058ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68732025-12-23T22:03:06.061ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68742025-12-23T22:03:06.061ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68752025-12-23T22:03:06.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68762025-12-23T22:03:06.063ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68772025-12-23T22:03:06.065ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68782025-12-23T22:03:06.065ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68792025-12-23T22:03:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
68802025-12-23T22:03:06.097ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
68812025-12-23T22:03:06.097ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
68822025-12-23T22:03:06.157ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c35d777-2419-4b06-aea5-5a6d40205a31", "content-length": "95980", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68832025-12-23T22:03:06.159ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68842025-12-23T22:03:06.159ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68852025-12-23T22:03:06.159ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
68862025-12-23T22:03:06.159ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
68872025-12-23T22:03:06.159ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
68882025-12-23T22:03:06.159ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
68892025-12-23T22:03:06.159ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
68902025-12-23T22:03:06.159ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
68912025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "035e2ec9-f440-4a20-b2e4-0039d342309e", "content-length": "64", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68922025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
68932025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "702d93f4-1614-4b84-ac79-c056763477f2", "content-length": "64", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68942025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "155949c4-ecc7-43e6-99c1-6ecbd44e9efa", "content-length": "64", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68952025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
68962025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0785507a-abb1-4279-8b2f-b12512010eda", "content-length": "64", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68972025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
68982025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c53e3bb7-30f6-482c-91b7-6ec05e4fbca5", "content-length": "64", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
68992025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
69002025-12-23T22:03:06.160ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
69012025-12-23T22:03:06.197ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6980f92-e84f-47c1-a663-80a9c7002d7b", "content-length": "418154", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69022025-12-23T22:03:06.198ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "be621bc2-0f35-4277-9e3d-09ca2a98c8c1", "content-length": "413744", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69032025-12-23T22:03:06.199ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d632b160-8800-47ca-b8d0-334fed90c9f1", "content-length": "411685", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69042025-12-23T22:03:06.199ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ad44364-3952-4623-9f4b-d7802fb97164", "content-length": "415827", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69052025-12-23T22:03:06.200ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2a02f24-393d-4e1a-888e-ca193e199f72", "content-length": "430649", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69062025-12-23T22:03:06.203ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
69072025-12-23T22:03:06.203ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
69082025-12-23T22:03:06.203ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
69092025-12-23T22:03:06.203ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
69102025-12-23T22:03:06.203ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
69112025-12-23T22:03:06.234ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f2c4b98-4b81-4370-8e91-c3c5f0bebc0c (omicron_zone)
69122025-12-23T22:03:06.234ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
69132025-12-23T22:03:06.237ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f87a5b1-71cb-4f16-8c6a-734b14acd282", "content-length": "129", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69142025-12-23T22:03:06.239ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
69152025-12-23T22:03:06.329ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
69162025-12-23T22:03:06.329ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
69172025-12-23T22:03:06.330ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "277e8f67-87ce-4a6c-852e-c2f6b55f5098", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69182025-12-23T22:03:06.330ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = a98c752e-5a58-41fa-9f53-a8d4f529a42f (omicron_zone)
69192025-12-23T22:03:06.332ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
69202025-12-23T22:03:06.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
69212025-12-23T22:03:06.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea946e8b-3821-45f6-993a-ce1150c87cc9", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69222025-12-23T22:03:06.336ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = e9c3ce58-81e4-4140-937f-1b169b78f235 (omicron_zone)
69232025-12-23T22:03:06.337ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
69242025-12-23T22:03:06.337ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
69252025-12-23T22:03:06.338ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c7fe02b-7b88-45fc-bbf0-c43a2d7192dd", "content-length": "9552", "date": "Tue, 23 Dec 2025 22:03:06 GMT"} })
69262025-12-23T22:03:06.338ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = f456c4c8-e5fd-45ce-a34c-a01d88dd14ad (omicron_zone)
69272025-12-23T22:03:06.339ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
69282025-12-23T22:03:06.339ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
69292025-12-23T22:03:06.340ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
69302025-12-23T22:03:06.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
69312025-12-23T22:03:06.343ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69322025-12-23T22:03:06.345ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
69332025-12-23T22:03:06.345ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
69342025-12-23T22:03:06.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
69352025-12-23T22:03:06.543ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 3c731628-f270-45f4-b971-80bb1924acd4 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
69362025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 3c731628-f270-45f4-b971-80bb1924acd4 time_started = 2025-12-23 22:03:06.046553 UTC
69372025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 47 reason = Dependency
69382025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 70 reason = Dependency
69392025-12-23T22:03:06.543ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
69402025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.95\u{b5}s iteration = 47 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
69412025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
69422025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 855.817348ms iteration = 17 status = {"collection_id":"3c731628-f270-45f4-b971-80bb1924acd4","time_done":"2025-12-23 22:03:06.339335 UTC","time_started":"2025-12-23 22:03:06.046553 UTC"}
69432025-12-23T22:03:06.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
69442025-12-23T22:03:06.545ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69452025-12-23T22:03:06.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
69462025-12-23T22:03:06.548ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
69472025-12-23T22:03:06.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
69482025-12-23T22:03:06.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
69492025-12-23T22:03:06.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
69502025-12-23T22:03:06.665ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 48 reason = Dependency
69512025-12-23T22:03:06.665ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 122.817896ms iteration = 70 status = {"Loaded":{"collection_id":"092721a8-1804-4f5c-a541-0145515b5278","time_loaded":"2025-12-23T22:03:06.543093989Z","time_started":"2025-12-23T22:03:06.048942Z"}}
69522025-12-23T22:03:06.665ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
69532025-12-23T22:03:06.665ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 44 reason = Dependency
69542025-12-23T22:03:06.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 20 reason = Dependency
69552025-12-23T22:03:06.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 33 reason = Dependency
69562025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 26.221\u{b5}s iteration = 48 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
69572025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69582025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
69592025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69602025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69612025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
69622025-12-23T22:03:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69632025-12-23T22:03:06.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69642025-12-23T22:03:06.670ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69652025-12-23T22:03:06.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69662025-12-23T22:03:06.673ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
69672025-12-23T22:03:06.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
69682025-12-23T22:03:06.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69692025-12-23T22:03:06.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69702025-12-23T22:03:06.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
69712025-12-23T22:03:06.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69722025-12-23T22:03:06.674ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
69732025-12-23T22:03:06.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69742025-12-23T22:03:06.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69752025-12-23T22:03:06.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69762025-12-23T22:03:06.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69772025-12-23T22:03:06.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69782025-12-23T22:03:06.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69792025-12-23T22:03:06.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69802025-12-23T22:03:06.679ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
69812025-12-23T22:03:06.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
69822025-12-23T22:03:06.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
69832025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
69842025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69852025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
69862025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69872025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69882025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69892025-12-23T22:03:06.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
69902025-12-23T22:03:06.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69912025-12-23T22:03:06.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69922025-12-23T22:03:06.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69932025-12-23T22:03:06.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69942025-12-23T22:03:06.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69952025-12-23T22:03:06.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
69962025-12-23T22:03:06.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
69972025-12-23T22:03:06.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
69982025-12-23T22:03:06.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69992025-12-23T22:03:06.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
70002025-12-23T22:03:06.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70012025-12-23T22:03:06.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70022025-12-23T22:03:06.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70032025-12-23T22:03:06.688ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 22.616728ms iteration = 44 status = {"physical_disks_added":0}
70042025-12-23T22:03:06.689ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
70052025-12-23T22:03:06.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
70062025-12-23T22:03:06.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
70072025-12-23T22:03:06.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
70082025-12-23T22:03:06.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70092025-12-23T22:03:06.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70102025-12-23T22:03:06.691ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
70112025-12-23T22:03:06.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
70122025-12-23T22:03:06.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
70132025-12-23T22:03:06.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70142025-12-23T22:03:06.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70152025-12-23T22:03:06.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70162025-12-23T22:03:06.694ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
70172025-12-23T22:03:06.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
70182025-12-23T22:03:06.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
70192025-12-23T22:03:06.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70202025-12-23T22:03:06.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70212025-12-23T22:03:06.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70222025-12-23T22:03:06.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70232025-12-23T22:03:06.697ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
70242025-12-23T22:03:06.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
70252025-12-23T22:03:06.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
70262025-12-23T22:03:06.701ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
70272025-12-23T22:03:06.701ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 35.227004ms iteration = 20 status = {"blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","inventory_collection_id":"092721a8-1804-4f5c-a541-0145515b5278","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
70282025-12-23T22:03:06.703ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70292025-12-23T22:03:06.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70302025-12-23T22:03:06.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70312025-12-23T22:03:06.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70322025-12-23T22:03:06.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70332025-12-23T22:03:06.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70342025-12-23T22:03:06.710ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
70352025-12-23T22:03:06.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70362025-12-23T22:03:06.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70372025-12-23T22:03:06.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70382025-12-23T22:03:06.714ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70392025-12-23T22:03:06.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70402025-12-23T22:03:06.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70412025-12-23T22:03:06.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70422025-12-23T22:03:06.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70432025-12-23T22:03:06.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70442025-12-23T22:03:06.721ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7b1eabb9-bf00-44f4-8a64-94c752911eb9, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
70452025-12-23T22:03:06.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70462025-12-23T22:03:06.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70472025-12-23T22:03:06.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70482025-12-23T22:03:06.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70492025-12-23T22:03:06.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70502025-12-23T22:03:06.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
70512025-12-23T22:03:06.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70522025-12-23T22:03:06.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70532025-12-23T22:03:06.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70542025-12-23T22:03:06.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70552025-12-23T22:03:06.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70562025-12-23T22:03:06.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
70572025-12-23T22:03:06.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70582025-12-23T22:03:06.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70592025-12-23T22:03:06.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70602025-12-23T22:03:06.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70612025-12-23T22:03:06.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70622025-12-23T22:03:06.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70632025-12-23T22:03:06.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70642025-12-23T22:03:06.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70652025-12-23T22:03:06.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70662025-12-23T22:03:06.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70672025-12-23T22:03:06.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70682025-12-23T22:03:06.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70692025-12-23T22:03:06.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70702025-12-23T22:03:06.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70712025-12-23T22:03:06.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70722025-12-23T22:03:06.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70732025-12-23T22:03:06.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70742025-12-23T22:03:06.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70752025-12-23T22:03:06.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70762025-12-23T22:03:06.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70772025-12-23T22:03:06.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70782025-12-23T22:03:06.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70792025-12-23T22:03:06.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70802025-12-23T22:03:06.772ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 7e57e16c-be5d-4c19-8f13-18f10f5f05b2, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
70812025-12-23T22:03:06.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70822025-12-23T22:03:06.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70832025-12-23T22:03:06.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70842025-12-23T22:03:06.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70852025-12-23T22:03:06.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70862025-12-23T22:03:06.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70872025-12-23T22:03:06.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70882025-12-23T22:03:06.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
70892025-12-23T22:03:06.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70902025-12-23T22:03:06.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70912025-12-23T22:03:06.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70922025-12-23T22:03:06.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70932025-12-23T22:03:06.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70942025-12-23T22:03:06.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
70952025-12-23T22:03:06.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70962025-12-23T22:03:06.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70972025-12-23T22:03:06.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
70982025-12-23T22:03:06.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
70992025-12-23T22:03:06.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71002025-12-23T22:03:06.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
71012025-12-23T22:03:06.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71022025-12-23T22:03:06.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71032025-12-23T22:03:06.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71042025-12-23T22:03:06.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71052025-12-23T22:03:06.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71062025-12-23T22:03:06.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
71072025-12-23T22:03:06.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71082025-12-23T22:03:06.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71092025-12-23T22:03:06.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71102025-12-23T22:03:06.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71112025-12-23T22:03:06.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71122025-12-23T22:03:06.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
71132025-12-23T22:03:06.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71142025-12-23T22:03:06.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71152025-12-23T22:03:06.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71162025-12-23T22:03:06.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71172025-12-23T22:03:06.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71182025-12-23T22:03:06.812ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
71192025-12-23T22:03:06.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71202025-12-23T22:03:06.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71212025-12-23T22:03:06.852ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
71222025-12-23T22:03:06.852ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
71232025-12-23T22:03:06.852ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 213c7cea-d9a3-4cce-8c16-5672dc25e043 parent_id = 02039deb-aee8-405c-80d5-e584bd199fe1 sled_id = ff85f534-787c-483e-ba89-a37cdea1dd38
71242025-12-23T22:03:06.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71252025-12-23T22:03:06.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71262025-12-23T22:03:06.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71272025-12-23T22:03:06.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
71282025-12-23T22:03:06.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
71292025-12-23T22:03:06.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
71302025-12-23T22:03:06.860ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
71312025-12-23T22:03:06.860ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 02039deb-aee8-405c-80d5-e584bd199fe1
71322025-12-23T22:03:06.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 194.791689ms iteration = 33 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"02039deb-aee8-405c-80d5-e584bd199fe1","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled ff85f534-787c-483e-ba89-a37cdea1dd38: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
71332025-12-23T22:03:09.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 158 reason = Timeout
71342025-12-23T22:03:09.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71352025-12-23T22:03:09.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71362025-12-23T22:03:09.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71372025-12-23T22:03:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
71382025-12-23T22:03:09.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71392025-12-23T22:03:09.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71402025-12-23T22:03:09.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
71412025-12-23T22:03:09.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.325087ms iteration = 158 status = {"config_updated":false}
71422025-12-23T22:03:11.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 432 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = 0c0fe928-20a1-4694-ad33-0466088be126 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
71432025-12-23T22:03:14.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
71442025-12-23T22:03:14.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71452025-12-23T22:03:14.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
71462025-12-23T22:03:14.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 159 reason = Timeout
71472025-12-23T22:03:14.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71482025-12-23T22:03:14.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71492025-12-23T22:03:14.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71502025-12-23T22:03:14.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71512025-12-23T22:03:14.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71522025-12-23T22:03:14.757ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71532025-12-23T22:03:14.758ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71542025-12-23T22:03:14.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71552025-12-23T22:03:14.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71562025-12-23T22:03:14.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.994473ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
71572025-12-23T22:03:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
71582025-12-23T22:03:14.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71592025-12-23T22:03:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71602025-12-23T22:03:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71612025-12-23T22:03:14.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71622025-12-23T22:03:14.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71632025-12-23T22:03:14.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
71642025-12-23T22:03:14.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.13037ms iteration = 159 status = {"config_updated":false}
71652025-12-23T22:03:14.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71662025-12-23T22:03:14.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71672025-12-23T22:03:14.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71682025-12-23T22:03:14.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
71692025-12-23T22:03:14.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71702025-12-23T22:03:14.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71712025-12-23T22:03:14.782ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
71722025-12-23T22:03:14.782ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.775576ms iteration = 80 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
71732025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 71 reason = Timeout
71742025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 54 reason = Timeout
71752025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 160 reason = Timeout
71762025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
71772025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
71782025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71792025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 54 reason = Timeout
71802025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
71812025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
71822025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
71832025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71842025-12-23T22:03:19.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
71852025-12-23T22:03:19.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
71862025-12-23T22:03:19.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
71872025-12-23T22:03:19.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71882025-12-23T22:03:19.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71892025-12-23T22:03:19.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71902025-12-23T22:03:19.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71912025-12-23T22:03:19.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
71922025-12-23T22:03:19.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
71932025-12-23T22:03:19.761ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
71942025-12-23T22:03:19.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
71952025-12-23T22:03:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
71962025-12-23T22:03:19.763ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
71972025-12-23T22:03:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
71982025-12-23T22:03:19.763ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71992025-12-23T22:03:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
72002025-12-23T22:03:19.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
72012025-12-23T22:03:19.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72022025-12-23T22:03:19.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.887912ms iteration = 54 status = "NoSitrep"
72032025-12-23T22:03:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72042025-12-23T22:03:19.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72052025-12-23T22:03:19.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72062025-12-23T22:03:19.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
72072025-12-23T22:03:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.946252ms iteration = 160 status = {"config_updated":false}
72082025-12-23T22:03:19.765ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 092721a8-1804-4f5c-a541-0145515b5278 original_time_started = 2025-12-23 22:03:06.048942 UTC
72092025-12-23T22:03:19.765ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.063376ms iteration = 71 status = {"Loaded":{"collection_id":"092721a8-1804-4f5c-a541-0145515b5278","time_loaded":"2025-12-23T22:03:19.756203898Z","time_started":"2025-12-23T22:03:06.048942Z"}}
72102025-12-23T22:03:19.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72112025-12-23T22:03:19.767ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
72122025-12-23T22:03:19.769ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
72132025-12-23T22:03:19.769ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
72142025-12-23T22:03:19.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.243194ms iteration = 54 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
72152025-12-23T22:03:21.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 504 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = edd3d743-fed5-4fe2-9898-6df30e247235 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
72162025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
72172025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72182025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
72192025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72202025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72212025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 161 reason = Timeout
72222025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
72232025-12-23T22:03:24.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
72242025-12-23T22:03:24.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
72252025-12-23T22:03:24.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72262025-12-23T22:03:24.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72272025-12-23T22:03:24.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72282025-12-23T22:03:24.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72292025-12-23T22:03:24.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.473557ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
72302025-12-23T22:03:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
72312025-12-23T22:03:24.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72322025-12-23T22:03:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
72332025-12-23T22:03:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
72342025-12-23T22:03:24.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
72352025-12-23T22:03:24.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
72362025-12-23T22:03:24.770ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
72372025-12-23T22:03:24.770ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.781668ms iteration = 161 status = {"config_updated":false}
72382025-12-23T22:03:24.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72392025-12-23T22:03:24.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
72402025-12-23T22:03:24.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72412025-12-23T22:03:24.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
72422025-12-23T22:03:24.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72432025-12-23T22:03:24.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
72442025-12-23T22:03:24.797ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
72452025-12-23T22:03:24.797ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.52228ms iteration = 81 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
72462025-12-23T22:03:29.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 162 reason = Timeout
72472025-12-23T22:03:29.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
72482025-12-23T22:03:29.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
72492025-12-23T22:03:29.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502025-12-23T22:03:29.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
72512025-12-23T22:03:29.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
72522025-12-23T22:03:29.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
72532025-12-23T22:03:29.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
72542025-12-23T22:03:29.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.373348ms iteration = 162 status = {"config_updated":false}
72552025-12-23T22:03:31.822ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 494 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = c2ad320f-3f7f-41e1-8539-f98eeae6ec26 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
72562025-12-23T22:03:34.620ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
72572025-12-23T22:03:34.620ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 28 reason = Timeout
72582025-12-23T22:03:34.620ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
72592025-12-23T22:03:34.620ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
72602025-12-23T22:03:34.620ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
72612025-12-23T22:03:34.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
72622025-12-23T22:03:34.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
72632025-12-23T22:03:34.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 723272523 } })
72642025-12-23T22:03:34.623ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 723447398 } })
72652025-12-23T22:03:34.624ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
72662025-12-23T22:03:34.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 723984153 } })
72672025-12-23T22:03:34.625ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.809206ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
72682025-12-23T22:03:34.688ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
72692025-12-23T22:03:34.690ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 69.576624ms iteration = 28 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
72702025-12-23T22:03:34.696ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
72712025-12-23T22:03:34.696ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
72722025-12-23T22:03:34.697ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "df6f0032-0bc8-49c2-97e5-8e480ae53c7c", "content-length": "26", "date": "Tue, 23 Dec 2025 22:03:34 GMT"} })
72732025-12-23T22:03:34.697ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
72742025-12-23T22:03:34.698ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
72752025-12-23T22:03:34.699ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 798695032 } })
72762025-12-23T22:03:34.745ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 125.025679ms iteration = 28 status = {}
72772025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
72782025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
72792025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
72802025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72812025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72822025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
72832025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
72842025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72852025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
72862025-12-23T22:03:34.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72872025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
72882025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
72892025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
72902025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72912025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
72922025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72932025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
72942025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
72952025-12-23T22:03:34.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
72962025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 28 reason = Timeout
72972025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 55 reason = Timeout
72982025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.011\u{b5}s iteration = 28 status = {"disabled":true,"errors":[],"sps":[]}
72992025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 45 reason = Timeout
73002025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 72 reason = Timeout
73012025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
73022025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
73032025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
73042025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 42 reason = Timeout
73052025-12-23T22:03:34.756ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
73062025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 55 reason = Timeout
73072025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
73082025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73092025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 163 reason = Timeout
73102025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
73112025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73122025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73132025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
73142025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73152025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
73162025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
73172025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
73182025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
73192025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
73202025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
73212025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
73222025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
73232025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
73242025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73252025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 42 reason = Timeout
73262025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
73272025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
73282025-12-23T22:03:34.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
73292025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73302025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
73312025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73322025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73332025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
73342025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
73352025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
73362025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
73372025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
73382025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
73392025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73402025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
73412025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73422025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.702117ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
73432025-12-23T22:03:34.757ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
73442025-12-23T22:03:34.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
73452025-12-23T22:03:34.758ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73462025-12-23T22:03:34.758ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73472025-12-23T22:03:34.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73482025-12-23T22:03:34.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73492025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73502025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73512025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73522025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73532025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73542025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73552025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73562025-12-23T22:03:34.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73572025-12-23T22:03:34.760ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73582025-12-23T22:03:34.760ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
73592025-12-23T22:03:34.760ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
73602025-12-23T22:03:34.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
73612025-12-23T22:03:34.760ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73622025-12-23T22:03:34.760ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 6.009449ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73632025-12-23T22:03:34.763ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
73642025-12-23T22:03:34.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
73652025-12-23T22:03:34.764ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
73662025-12-23T22:03:34.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73672025-12-23T22:03:34.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73682025-12-23T22:03:34.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73692025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
73702025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
73712025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
73722025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73732025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
73742025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
73752025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
73762025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73772025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
73782025-12-23T22:03:34.766ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
73792025-12-23T22:03:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73802025-12-23T22:03:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
73812025-12-23T22:03:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
73822025-12-23T22:03:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73832025-12-23T22:03:34.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
73842025-12-23T22:03:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
73852025-12-23T22:03:34.768ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
73862025-12-23T22:03:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
73872025-12-23T22:03:34.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
73882025-12-23T22:03:34.768ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73892025-12-23T22:03:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73902025-12-23T22:03:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: ff85f534-787c-483e-ba89-a37cdea1dd38 (sled), lookup_type: ById(ff85f534-787c-483e-ba89-a37cdea1dd38) } result = Ok(())
73912025-12-23T22:03:34.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73922025-12-23T22:03:34.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73932025-12-23T22:03:34.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73942025-12-23T22:03:34.770ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 13.856481ms iteration = 55 status = "NoSitrep"
73952025-12-23T22:03:34.771ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
73962025-12-23T22:03:34.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
73972025-12-23T22:03:34.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73982025-12-23T22:03:34.772ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 092721a8-1804-4f5c-a541-0145515b5278 original_time_started = 2025-12-23 22:03:06.048942 UTC
73992025-12-23T22:03:34.772ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 16.079944ms iteration = 72 status = {"Loaded":{"collection_id":"092721a8-1804-4f5c-a541-0145515b5278","time_loaded":"2025-12-23T22:03:34.756447465Z","time_started":"2025-12-23T22:03:06.048942Z"}}
74002025-12-23T22:03:34.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74012025-12-23T22:03:34.773ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 18.200614ms iteration = 28 status = 0
74022025-12-23T22:03:34.773ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 873084282 } })
74032025-12-23T22:03:34.773ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74042025-12-23T22:03:34.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74052025-12-23T22:03:34.774ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
74062025-12-23T22:03:34.774ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74072025-12-23T22:03:34.774ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
74082025-12-23T22:03:34.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74092025-12-23T22:03:34.775ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74102025-12-23T22:03:34.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74112025-12-23T22:03:34.775ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 875170341 } })
74122025-12-23T22:03:34.775ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
74132025-12-23T22:03:34.775ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 19.205122ms iteration = 163 status = {"config_updated":false}
74142025-12-23T22:03:34.776ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74152025-12-23T22:03:34.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74162025-12-23T22:03:34.779ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74172025-12-23T22:03:34.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
74182025-12-23T22:03:34.780ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
74192025-12-23T22:03:34.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
74202025-12-23T22:03:34.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
74212025-12-23T22:03:34.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74222025-12-23T22:03:34.782ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
74232025-12-23T22:03:34.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74242025-12-23T22:03:34.783ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
74252025-12-23T22:03:34.783ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 26.378644ms iteration = 42 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
74262025-12-23T22:03:34.784ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
74272025-12-23T22:03:34.784ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
74282025-12-23T22:03:34.784ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 27.541577ms iteration = 55 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
74292025-12-23T22:03:34.809ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
74302025-12-23T22:03:34.809ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.56481ms iteration = 82 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
74312025-12-23T22:03:34.822ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74322025-12-23T22:03:34.822ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
74332025-12-23T22:03:34.822ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
74342025-12-23T22:03:34.822ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
74352025-12-23T22:03:34.823ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.731124ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
74362025-12-23T22:03:34.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
74372025-12-23T22:03:34.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
74382025-12-23T22:03:34.826ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74392025-12-23T22:03:34.826ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
74402025-12-23T22:03:34.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74412025-12-23T22:03:34.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
74422025-12-23T22:03:34.826ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 70.266453ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
74432025-12-23T22:03:34.827ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74442025-12-23T22:03:34.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
74452025-12-23T22:03:34.828ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 73.525805ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
74462025-12-23T22:03:34.828ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74472025-12-23T22:03:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74482025-12-23T22:03:34.829ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
74492025-12-23T22:03:34.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
74502025-12-23T22:03:34.830ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74512025-12-23T22:03:34.831ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 75.248764ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
74522025-12-23T22:03:34.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
74532025-12-23T22:03:34.833ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
74542025-12-23T22:03:34.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
74552025-12-23T22:03:34.844ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
74562025-12-23T22:03:34.844ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 88.044855ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
74572025-12-23T22:03:34.846ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f zone_address = fd00:1122:3344:101::2
74582025-12-23T22:03:34.846ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
74592025-12-23T22:03:34.847ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "23bf1117-ac50-4447-b9db-a05f3b256a17", "content-length": "26", "date": "Tue, 23 Dec 2025 22:03:34 GMT"} })
74602025-12-23T22:03:34.847ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74612025-12-23T22:03:34.849ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
74622025-12-23T22:03:34.857ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
74632025-12-23T22:03:34.857ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
74642025-12-23T22:03:34.857ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc92009e-6e8b-4976-b29e-e46dd05f8d0b", "content-length": "372", "date": "Tue, 23 Dec 2025 22:03:34 GMT"} })
74652025-12-23T22:03:34.857ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74662025-12-23T22:03:34.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74672025-12-23T22:03:34.873ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 116.993973ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
74682025-12-23T22:03:34.883ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 126.547102ms iteration = 45 status = {"physical_disks_added":0}
74692025-12-23T22:03:34.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1712, tv_nsec: 983334434 } })
74702025-12-23T22:03:34.885ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
74712025-12-23T22:03:34.885ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
74722025-12-23T22:03:34.885ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 130.238007ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
74732025-12-23T22:03:34.913ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
74742025-12-23T22:03:34.913ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
74752025-12-23T22:03:34.913ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "611a571a-3e2f-4b49-8834-127b3104dcd6", "date": "Tue, 23 Dec 2025 22:03:34 GMT"} })
74762025-12-23T22:03:34.913ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74772025-12-23T22:03:34.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74782025-12-23T22:03:34.916ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74792025-12-23T22:03:34.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74802025-12-23T22:03:34.917ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74812025-12-23T22:03:34.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74822025-12-23T22:03:34.919ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-ff85f534-787c-483e-ba89-a37cdea1dd38.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1713, tv_nsec: 18658111 } })
74832025-12-23T22:03:34.919ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74842025-12-23T22:03:34.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74852025-12-23T22:03:34.922ZINFOnexus (BackgroundTasks): VPC route manager sled ff85f534-787c-483e-ba89-a37cdea1dd38 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
74862025-12-23T22:03:34.922ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
74872025-12-23T22:03:34.924ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74882025-12-23T22:03:34.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74892025-12-23T22:03:34.926ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74902025-12-23T22:03:34.928ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fd9a86b0-fd3b-4b1c-b2fd-d13a6766c227", "date": "Tue, 23 Dec 2025 22:03:34 GMT"} })
74912025-12-23T22:03:34.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
74922025-12-23T22:03:34.928ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74932025-12-23T22:03:34.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74942025-12-23T22:03:34.930ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 173.771106ms iteration = 42 status = {}
74952025-12-23T22:03:34.930ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74962025-12-23T22:03:34.934ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
74972025-12-23T22:03:34.934ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
74982025-12-23T22:03:34.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
74992025-12-23T22:03:34.953ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75002025-12-23T22:03:34.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75012025-12-23T22:03:34.954ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: e822edf4-325b-4f51-b446-34c3e004bf98, rack_id: 411517f7-7968-4283-8f9d-7c862fc2242f, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(ce5c3555-5dbb-4897-8694-ca15205e31e7) } rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
75022025-12-23T22:03:34.954ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75032025-12-23T22:03:34.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75042025-12-23T22:03:34.967ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
75052025-12-23T22:03:34.967ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = ce5c3555-5dbb-4897-8694-ca15205e31e7
75062025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
75072025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
75082025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
75092025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
75102025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
75112025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
75122025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
75132025-12-23T22:03:34.968ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
75142025-12-23T22:03:34.969ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f
75152025-12-23T22:03:34.969ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75162025-12-23T22:03:34.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75172025-12-23T22:03:34.971ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch_location = Switch0
75182025-12-23T22:03:34.971ZDEBGnexus (BackgroundTasks): client request SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
75192025-12-23T22:03:34.971ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
75202025-12-23T22:03:34.972ZDEBGnexus (BackgroundTasks): client response SledAgent = ff85f534-787c-483e-ba89-a37cdea1dd38 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "85a70a17-7722-4c01-808f-d11c11ec5895", "content-length": "691", "date": "Tue, 23 Dec 2025 22:03:34 GMT"} })
75212025-12-23T22:03:34.972ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
75222025-12-23T22:03:34.972ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
75232025-12-23T22:03:34.972ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
75242025-12-23T22:03:34.974ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 217.702445ms iteration = 28 status = {}
75252025-12-23T22:03:35.246ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8e19d046-034d-415c-bbeb-e4bb5ce210d5", "date": "Tue, 23 Dec 2025 22:03:35 GMT"} })
75262025-12-23T22:03:35.246ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f switch = Switch0
75272025-12-23T22:03:35.246ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
75282025-12-23T22:03:35.247ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 411517f7-7968-4283-8f9d-7c862fc2242f result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "791fbfb7-ea8a-40f3-b371-9e1f314a86ae", "content-length": "682", "date": "Tue, 23 Dec 2025 22:03:35 GMT"} })
75292025-12-23T22:03:35.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75302025-12-23T22:03:35.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75312025-12-23T22:03:35.251ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75322025-12-23T22:03:35.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75332025-12-23T22:03:35.255ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75342025-12-23T22:03:35.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75352025-12-23T22:03:35.258ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75362025-12-23T22:03:35.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75372025-12-23T22:03:35.265ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 509.377069ms iteration = 28 status = {}
75382025-12-23T22:03:35.719ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 5
75392025-12-23T22:03:35.719ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::c]:12221
75402025-12-23T22:03:35.749ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
75412025-12-23T22:03:35.749ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:52401
75422025-12-23T22:03:35.749ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52401 req_id = e40001de-85f1-41dc-a6cc-f4b9363399f1 uri = /metrics/producers version = 1.0.0
75432025-12-23T22:03:35.749ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52401 req_id = e40001de-85f1-41dc-a6cc-f4b9363399f1 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
75442025-12-23T22:03:35.749ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52401 req_id = e40001de-85f1-41dc-a6cc-f4b9363399f1 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
75452025-12-23T22:03:35.803ZINFOnexus: assigned collector to new producer collector_id = f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f file = nexus/src/app/oximeter.rs:76 producer_id = 8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
75462025-12-23T22:03:35.803ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 54096 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:52401 req_id = e40001de-85f1-41dc-a6cc-f4b9363399f1 response_code = 201 uri = /metrics/producers
75472025-12-23T22:03:35.803ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "e40001de-85f1-41dc-a6cc-f4b9363399f1", "content-length": "41", "date": "Tue, 23 Dec 2025 22:03:35 GMT"} })
75482025-12-23T22:03:35.805ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
75492025-12-23T22:03:39.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 164 reason = Timeout
75502025-12-23T22:03:39.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75512025-12-23T22:03:39.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75522025-12-23T22:03:39.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75532025-12-23T22:03:39.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
75542025-12-23T22:03:39.762ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75552025-12-23T22:03:39.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75562025-12-23T22:03:39.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
75572025-12-23T22:03:39.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.517393ms iteration = 164 status = {"config_updated":false}
75582025-12-23T22:03:41.816ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:63420
75592025-12-23T22:03:41.816ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = 80b57aac-2603-4f7c-8441-2b1e337e3dce uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending version = 1.0.0
75602025-12-23T22:03:41.816ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = 80b57aac-2603-4f7c-8441-2b1e337e3dce roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
75612025-12-23T22:03:41.816ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = 80b57aac-2603-4f7c-8441-2b1e337e3dce resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
75622025-12-23T22:03:41.819ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3086 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = 80b57aac-2603-4f7c-8441-2b1e337e3dce response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?sort_by=id_ascending
75632025-12-23T22:03:41.819ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = a623880f-980a-49e3-8de3-ee8f09c18be2 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D version = 1.0.0
75642025-12-23T22:03:41.819ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = a623880f-980a-49e3-8de3-ee8f09c18be2 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
75652025-12-23T22:03:41.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = a623880f-980a-49e3-8de3-ee8f09c18be2 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
75662025-12-23T22:03:41.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3057 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63420 req_id = a623880f-980a-49e3-8de3-ee8f09c18be2 response_code = 200 uri = /metrics/collectors/f14ab8a8-91ef-42b8-b038-1fc4c5c3ae6f/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmZjg1ZjUzNC03ODdjLTQ4M2UtYmE4OS1hMzdjZGVhMWRkMzgifX0%3D
75672025-12-23T22:03:41.823ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 429 local_addr = [fd00:1122:3344:101::c]:33363 method = GET remote_addr = [fd00:1122:3344:101::d]:39102 req_id = c61df636-65cd-473c-a211-f157054720f2 response_code = 200 uri = /8e593c8b-acdb-44d0-8ac2-98e0fc1adb21
75682025-12-23T22:03:44.755ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
75692025-12-23T22:03:44.755ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
75702025-12-23T22:03:44.755ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75712025-12-23T22:03:44.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75722025-12-23T22:03:44.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75732025-12-23T22:03:44.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 165 reason = Timeout
75742025-12-23T22:03:44.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75752025-12-23T22:03:44.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75762025-12-23T22:03:44.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75772025-12-23T22:03:44.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75782025-12-23T22:03:44.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75792025-12-23T22:03:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75802025-12-23T22:03:44.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75812025-12-23T22:03:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75822025-12-23T22:03:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
75832025-12-23T22:03:44.761ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75842025-12-23T22:03:44.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75852025-12-23T22:03:44.763ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
75862025-12-23T22:03:44.763ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.375118ms iteration = 165 status = {"config_updated":false}
75872025-12-23T22:03:44.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75882025-12-23T22:03:44.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75892025-12-23T22:03:44.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75902025-12-23T22:03:44.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 02039deb-aee8-405c-80d5-e584bd199fe1, lookup_type: ById(02039deb-aee8-405c-80d5-e584bd199fe1) } result = Ok(())
75912025-12-23T22:03:44.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75922025-12-23T22:03:44.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75932025-12-23T22:03:44.790ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 original_time_created = 2025-12-23 21:49:51.527369 UTC target_id = 02039deb-aee8-405c-80d5-e584bd199fe1 time_created = 2025-12-23 21:49:51.527369 UTC
75942025-12-23T22:03:44.790ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.979868ms iteration = 83 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"02039deb-aee8-405c-80d5-e584bd199fe1","time_created":"2025-12-23T21:49:51.527369Z"}
75952025-12-23T22:03:44.802ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75962025-12-23T22:03:44.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75972025-12-23T22:03:44.840ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 85.196245ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75982025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 56 reason = Timeout
75992025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 56 reason = Timeout
76002025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76012025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
76022025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 166 reason = Timeout
76032025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 73 reason = Timeout
76042025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
76052025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
76062025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
76072025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76082025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
76092025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
76102025-12-23T22:03:49.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76112025-12-23T22:03:49.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76122025-12-23T22:03:49.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76132025-12-23T22:03:49.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76142025-12-23T22:03:49.758ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76152025-12-23T22:03:49.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76162025-12-23T22:03:49.761ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
76172025-12-23T22:03:49.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
76182025-12-23T22:03:49.762ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
76192025-12-23T22:03:49.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
76202025-12-23T22:03:49.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
76212025-12-23T22:03:49.762ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
76222025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
76232025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
76242025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
76252025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
76262025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76272025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76282025-12-23T22:03:49.763ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.886912ms iteration = 56 status = "NoSitrep"
76292025-12-23T22:03:49.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76302025-12-23T22:03:49.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76312025-12-23T22:03:49.764ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 092721a8-1804-4f5c-a541-0145515b5278 original_time_started = 2025-12-23 22:03:06.048942 UTC
76322025-12-23T22:03:49.764ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.613423ms iteration = 73 status = {"Loaded":{"collection_id":"092721a8-1804-4f5c-a541-0145515b5278","time_loaded":"2025-12-23T22:03:49.756131975Z","time_started":"2025-12-23T22:03:06.048942Z"}}
76332025-12-23T22:03:49.764ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
76342025-12-23T22:03:49.764ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.664885ms iteration = 166 status = {"config_updated":false}
76352025-12-23T22:03:49.765ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76362025-12-23T22:03:49.767ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
76372025-12-23T22:03:49.769ZDEBGnexus (BackgroundTasks): deleted 0 of 0 sitreps sitreps background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 ids = [] sitreps_deleted = 0
76382025-12-23T22:03:49.769ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
76392025-12-23T22:03:49.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.085979ms iteration = 56 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}