Max level shown:
12024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
22024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
32024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
62024-08-06T04:30:09.054ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
72024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
82024-08-06T04:30:09.054ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
92024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
102024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
112024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
122024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
132024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
142024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
152024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
162024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
172024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
182024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
192024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
202024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
212024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
222024-08-06T04:30:09.054ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
232024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
242024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
252024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
262024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
272024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
282024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
292024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
302024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
312024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
322024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
332024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
342024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
352024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
362024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
372024-08-06T04:30:09.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
382024-08-06T04:30:09.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
392024-08-06T04:30:09.056ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
402024-08-06T04:30:09.057ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
412024-08-06T04:30:09.057ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
422024-08-06T04:30:09.057ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
432024-08-06T04:30:09.058ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
442024-08-06T04:30:09.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
452024-08-06T04:30:09.058ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.782098ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
462024-08-06T04:30:09.058ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
472024-08-06T04:30:09.058ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1205, tv_nsec: 146904136 } })
482024-08-06T04:30:09.058ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
492024-08-06T04:30:09.059ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
502024-08-06T04:30:09.059ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.017557ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
512024-08-06T04:30:09.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
522024-08-06T04:30:09.059ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
532024-08-06T04:30:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
542024-08-06T04:30:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
552024-08-06T04:30:09.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
562024-08-06T04:30:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
572024-08-06T04:30:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
582024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
592024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
602024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
612024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
622024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
632024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
642024-08-06T04:30:09.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
652024-08-06T04:30:09.061ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1205, tv_nsec: 150495232 } })
662024-08-06T04:30:09.062ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1205, tv_nsec: 150692903 } })
672024-08-06T04:30:09.062ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
682024-08-06T04:30:09.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
692024-08-06T04:30:09.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
702024-08-06T04:30:09.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
712024-08-06T04:30:09.062ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
722024-08-06T04:30:09.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
732024-08-06T04:30:09.062ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
742024-08-06T04:30:09.063ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.661583ms iteration = 12 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
752024-08-06T04:30:09.063ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
762024-08-06T04:30:09.063ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
772024-08-06T04:30:09.063ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
782024-08-06T04:30:09.063ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
792024-08-06T04:30:09.063ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
802024-08-06T04:30:09.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
812024-08-06T04:30:09.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
822024-08-06T04:30:09.065ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
832024-08-06T04:30:09.065ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
842024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
852024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
862024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
872024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
882024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
892024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
902024-08-06T04:30:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
912024-08-06T04:30:09.067ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
922024-08-06T04:30:09.067ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
932024-08-06T04:30:09.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
942024-08-06T04:30:09.067ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
952024-08-06T04:30:09.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
962024-08-06T04:30:09.068ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
972024-08-06T04:30:09.068ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.7918ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
982024-08-06T04:30:09.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
992024-08-06T04:30:09.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1002024-08-06T04:30:09.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1012024-08-06T04:30:09.070ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
1022024-08-06T04:30:09.070ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.318819ms iteration = 13 status = {"physical_disks_added":0}
1032024-08-06T04:30:09.074ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
1042024-08-06T04:30:09.074ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.892175ms iteration = 34 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
1052024-08-06T04:30:09.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1062024-08-06T04:30:09.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1072024-08-06T04:30:09.076ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1082024-08-06T04:30:09.078ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1092024-08-06T04:30:09.078ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1102024-08-06T04:30:09.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1112024-08-06T04:30:09.079ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1122024-08-06T04:30:09.079ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1132024-08-06T04:30:09.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1142024-08-06T04:30:09.081ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1152024-08-06T04:30:09.081ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1162024-08-06T04:30:09.081ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1172024-08-06T04:30:09.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1182024-08-06T04:30:09.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1192024-08-06T04:30:09.082ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1202024-08-06T04:30:09.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 76e61613-583c-43ed-85e5-238a1f2bdaaf, lookup_type: ById(76e61613-583c-43ed-85e5-238a1f2bdaaf) } result = Ok(())
1212024-08-06T04:30:09.085ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1222024-08-06T04:30:09.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1232024-08-06T04:30:09.086ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 32.123803ms iteration = 12 status = 0
1242024-08-06T04:30:09.208ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1252024-08-06T04:30:09.209ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "675d46ff-0e3b-4a03-99c9-b2c8ad7871c4", "content-length": "372", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1262024-08-06T04:30:09.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1272024-08-06T04:30:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1282024-08-06T04:30:09.210ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1292024-08-06T04:30:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1302024-08-06T04:30:09.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1312024-08-06T04:30:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1322024-08-06T04:30:09.215ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1332024-08-06T04:30:09.215ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "78699d39-9a12-485e-a6b2-95d1c1df8317", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1342024-08-06T04:30:09.217ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 162.732291ms iteration = 12 status = {}
1352024-08-06T04:30:09.296ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
1362024-08-06T04:30:09.296ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1372024-08-06T04:30:09.296ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b7905da1-67d0-40f8-9fe9-3cc78e62802e", "content-length": "26", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1382024-08-06T04:30:09.296ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1392024-08-06T04:30:09.298ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1402024-08-06T04:30:09.315ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1412024-08-06T04:30:09.316ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9b3f8fb8-8a41-4c22-bb36-b05346a77bfa", "content-length": "691", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1422024-08-06T04:30:09.316ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
1432024-08-06T04:30:09.316ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
1442024-08-06T04:30:09.316ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
1452024-08-06T04:30:09.318ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 263.928466ms iteration = 12 status = {}
1462024-08-06T04:30:09.322ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
1472024-08-06T04:30:09.322ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1482024-08-06T04:30:09.322ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "535d79cf-9bd2-40c2-b2ab-54c46b32a33d", "content-length": "26", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1492024-08-06T04:30:09.322ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1502024-08-06T04:30:09.324ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f zone_address = fd00:1122:3344:101::2
1512024-08-06T04:30:09.324ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1522024-08-06T04:30:09.324ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b3042640-d9ec-4d46-a93d-c11edde1b4de", "content-length": "26", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1532024-08-06T04:30:09.324ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1542024-08-06T04:30:09.324ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1552024-08-06T04:30:09.326ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
1562024-08-06T04:30:09.384ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1572024-08-06T04:30:09.384ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6d15214c-306f-4c7f-8349-6c8d63e53240", "content-length": "1", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1582024-08-06T04:30:09.386ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1592024-08-06T04:30:09.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1602024-08-06T04:30:09.387ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 333.567793ms iteration = 12 status = null
1612024-08-06T04:30:09.478ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 424.386801ms iteration = 12 status = {}
1622024-08-06T04:30:09.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1632024-08-06T04:30:09.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1642024-08-06T04:30:09.535ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: c3fd3796-62f1-488f-a5ed-505375c3218a, rack_id: ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e29905d0-96f1-4b4c-be22-f8c46aedb308) } rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0
1652024-08-06T04:30:09.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1662024-08-06T04:30:09.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1672024-08-06T04:30:09.543ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1682024-08-06T04:30:09.543ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7d426ea8-8484-4835-a870-78b99d21d7bb", "content-length": "121", "date": "Tue, 06 Aug 2024 04:30:09 GMT"} })
1692024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
1702024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e29905d0-96f1-4b4c-be22-f8c46aedb308
1712024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
1722024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
1732024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {}
1742024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {}
1752024-08-06T04:30:09.543ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
1762024-08-06T04:30:09.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1772024-08-06T04:30:09.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1782024-08-06T04:30:09.545ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1792024-08-06T04:30:09.546ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "55698d5b-a2d7-4a16-9c28-5ab0e61dd06c", "content-length": "2", "date": "Tue, 06 Aug 2024 04:30:09 GMT"} })
1802024-08-06T04:30:09.546ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1812024-08-06T04:30:09.546ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b395ae91-b660-4a1c-a960-63a7a3389c51", "content-length": "52", "date": "Tue, 06 Aug 2024 04:30:09 GMT"} })
1822024-08-06T04:30:09.546ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0
1832024-08-06T04:30:09.546ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1842024-08-06T04:30:09.590ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ed03d451-986b-4dc6-9f34-4eaad479918c", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1852024-08-06T04:30:09.590ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch = Switch0
1862024-08-06T04:30:09.590ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1872024-08-06T04:30:09.590ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "988b3471-8ce9-49f9-a2ff-2f1a8b13fafd", "content-length": "494", "date": "Tue, 06 Aug 2024 04:30:08 GMT"} })
1882024-08-06T04:30:09.590ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1892024-08-06T04:30:09.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1902024-08-06T04:30:09.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1912024-08-06T04:30:09.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1922024-08-06T04:30:09.595ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1932024-08-06T04:30:09.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1942024-08-06T04:30:09.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1952024-08-06T04:30:09.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1962024-08-06T04:30:09.604ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 549.819811ms iteration = 12 status = {}
1972024-08-06T04:30:09.807ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 355 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = 28de1a5d-6893-40ab-a7e6-722f1ffcf409 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
1982024-08-06T04:30:19.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
1992024-08-06T04:30:19.054ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
2002024-08-06T04:30:19.054ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2012024-08-06T04:30:19.054ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
2022024-08-06T04:30:19.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2032024-08-06T04:30:19.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2042024-08-06T04:30:19.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2052024-08-06T04:30:19.056ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2062024-08-06T04:30:19.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2072024-08-06T04:30:19.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2082024-08-06T04:30:19.057ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
2092024-08-06T04:30:19.057ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.013212ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2102024-08-06T04:30:19.058ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2112024-08-06T04:30:19.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2122024-08-06T04:30:19.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2132024-08-06T04:30:19.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2142024-08-06T04:30:19.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2152024-08-06T04:30:19.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2162024-08-06T04:30:19.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
2172024-08-06T04:30:19.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2182024-08-06T04:30:19.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2192024-08-06T04:30:19.074ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
2202024-08-06T04:30:19.074ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.177526ms iteration = 35 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
2212024-08-06T04:30:19.806ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 339 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = 4e098d1d-33c4-42fd-8e76-eededf13a199 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
2222024-08-06T04:30:29.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
2232024-08-06T04:30:29.054ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
2242024-08-06T04:30:29.054ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2252024-08-06T04:30:29.054ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
2262024-08-06T04:30:29.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2272024-08-06T04:30:29.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2282024-08-06T04:30:29.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2292024-08-06T04:30:29.056ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2302024-08-06T04:30:29.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2312024-08-06T04:30:29.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2322024-08-06T04:30:29.057ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
2332024-08-06T04:30:29.057ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.219203ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2342024-08-06T04:30:29.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2352024-08-06T04:30:29.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2362024-08-06T04:30:29.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2372024-08-06T04:30:29.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2382024-08-06T04:30:29.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2392024-08-06T04:30:29.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2402024-08-06T04:30:29.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
2412024-08-06T04:30:29.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2422024-08-06T04:30:29.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2432024-08-06T04:30:29.071ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
2442024-08-06T04:30:29.071ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.905893ms iteration = 36 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
2452024-08-06T04:30:29.807ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1061 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = 190ea655-a45f-46fa-844a-65b2253aabbf response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
2462024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
2472024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
2482024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2492024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
2502024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
2512024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
2522024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2532024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2542024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2552024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
2562024-08-06T04:30:39.054ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
2572024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
2582024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
2592024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2602024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
2612024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
2622024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2632024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2642024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
2652024-08-06T04:30:39.054ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:69 reason = blueprint disabled target_id = d234c62d-4359-4882-b5c9-00f71d500cb3
2662024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
2672024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.061\u{b5}s iteration = 8 status = {"error":"blueprint disabled","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3"}
2682024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2692024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2702024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
2712024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
2722024-08-06T04:30:39.054ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
2732024-08-06T04:30:39.054ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-08-06 04:20:39.054429450 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2742024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2752024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2762024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
2772024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
2782024-08-06T04:30:39.054ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
2792024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
2802024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
2812024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2822024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2832024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
2842024-08-06T04:30:39.054ZINFOnexus (BackgroundTasks): lookup region port task started background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:56
2852024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2862024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
2872024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
2882024-08-06T04:30:39.054ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
2892024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2902024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
2912024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
2922024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2932024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
2942024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
2952024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2962024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2972024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
2982024-08-06T04:30:39.054ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
2992024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3002024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
3012024-08-06T04:30:39.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3022024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3032024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3042024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3052024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3062024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3072024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3082024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3092024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3102024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3112024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3122024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3132024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3142024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3152024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3162024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3172024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3182024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3192024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3202024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3212024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3222024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3232024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3242024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3252024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3262024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3272024-08-06T04:30:39.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3282024-08-06T04:30:39.056ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1fd84097-e9d0-4dcf-b9a6-cd5ef4a17050.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3ed29dcb-ec9e-4bb3-a077-971eeaf403c4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("3ed29dcb-ec9e-4bb3-a077-971eeaf403c4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 143990284 } })
3292024-08-06T04:30:39.056ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("90c50f5b-823e-44aa-af6d-b7141d938159.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ad080ed5-9ffa-48a1-b545-93d912c0f457.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("fd3a9b48-dd1f-4ffd-9c2f-993172b6f67b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("fd3a9b48-dd1f-4ffd-9c2f-993172b6f67b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 144079125 } })
3302024-08-06T04:30:39.057ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3312024-08-06T04:30:39.057ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3322024-08-06T04:30:39.057ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3332024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3342024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3352024-08-06T04:30:39.058ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
3362024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.713077ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3372024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3382024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3392024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
3402024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.939928ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
3412024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3422024-08-06T04:30:39.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3432024-08-06T04:30:39.059ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 147606400 } })
3442024-08-06T04:30:39.059ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
3452024-08-06T04:30:39.059ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.188838ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
3462024-08-06T04:30:39.059ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3472024-08-06T04:30:39.060ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3482024-08-06T04:30:39.060ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
3492024-08-06T04:30:39.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3502024-08-06T04:30:39.061ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3512024-08-06T04:30:39.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3522024-08-06T04:30:39.061ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3532024-08-06T04:30:39.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3542024-08-06T04:30:39.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3552024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3562024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3572024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3582024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3592024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3602024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3612024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3622024-08-06T04:30:39.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3632024-08-06T04:30:39.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3642024-08-06T04:30:39.063ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3652024-08-06T04:30:39.063ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3662024-08-06T04:30:39.063ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
3672024-08-06T04:30:39.063ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3682024-08-06T04:30:39.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3692024-08-06T04:30:39.063ZINFOnexus (BackgroundTasks): lookup region port task done background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:150
3702024-08-06T04:30:39.063ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.051426ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
3712024-08-06T04:30:39.064ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 151985812 } })
3722024-08-06T04:30:39.064ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
3732024-08-06T04:30:39.064ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3742024-08-06T04:30:39.064ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3752024-08-06T04:30:39.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3762024-08-06T04:30:39.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3772024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
3782024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3792024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
3802024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3812024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3822024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3832024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3842024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3852024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3862024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3872024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3882024-08-06T04:30:39.066ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 154701772 } })
3892024-08-06T04:30:39.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3902024-08-06T04:30:39.067ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
3912024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3922024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3932024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3942024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
3952024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
3962024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
3972024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3982024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3992024-08-06T04:30:39.067ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4002024-08-06T04:30:39.068ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4012024-08-06T04:30:39.068ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4022024-08-06T04:30:39.069ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
4032024-08-06T04:30:39.069ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
4042024-08-06T04:30:39.069ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 14.516606ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
4052024-08-06T04:30:39.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4062024-08-06T04:30:39.070ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4072024-08-06T04:30:39.070ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4082024-08-06T04:30:39.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4092024-08-06T04:30:39.070ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4102024-08-06T04:30:39.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4112024-08-06T04:30:39.071ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
4122024-08-06T04:30:39.071ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 17.012943ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4132024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4142024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4152024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4162024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4172024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4182024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4192024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4202024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4212024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4222024-08-06T04:30:39.072ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4232024-08-06T04:30:39.073ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4242024-08-06T04:30:39.073ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 58777c7a-934b-4e4f-b565-dbe3cf772e2a dns_zone_name = oxide.test found_so_far = 0 version = 2
4252024-08-06T04:30:39.073ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4262024-08-06T04:30:39.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4272024-08-06T04:30:39.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4282024-08-06T04:30:39.074ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4292024-08-06T04:30:39.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4302024-08-06T04:30:39.074ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4312024-08-06T04:30:39.075ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4322024-08-06T04:30:39.075ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4332024-08-06T04:30:39.075ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
4342024-08-06T04:30:39.075ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.268744ms iteration = 14 status = {"physical_disks_added":0}
4352024-08-06T04:30:39.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4362024-08-06T04:30:39.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4372024-08-06T04:30:39.076ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4382024-08-06T04:30:39.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4392024-08-06T04:30:39.076ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4402024-08-06T04:30:39.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4412024-08-06T04:30:39.078ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4422024-08-06T04:30:39.078ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4432024-08-06T04:30:39.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4442024-08-06T04:30:39.078ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4452024-08-06T04:30:39.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4462024-08-06T04:30:39.078ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4472024-08-06T04:30:39.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4482024-08-06T04:30:39.079ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 58777c7a-934b-4e4f-b565-dbe3cf772e2a dns_zone_name = oxide.test found_so_far = 1 version = 2
4492024-08-06T04:30:39.079ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4502024-08-06T04:30:39.079ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-08-06 04:24:54.626419 UTC dns_group = external generation = 2
4512024-08-06T04:30:39.079ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.306554ms iteration = 7 status = {"generation":2}
4522024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4532024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4542024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4552024-08-06T04:30:39.080ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
4562024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 25.886148ms iteration = 13 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
4572024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
4582024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 14f1be63-b272-4578-a0ca-d8087b696597 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
4592024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4602024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4612024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4622024-08-06T04:30:39.080ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4632024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4642024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4652024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4662024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4672024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4682024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4692024-08-06T04:30:39.081ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 27.60335ms iteration = 4 status = {"nsuccess":5}
4702024-08-06T04:30:39.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4712024-08-06T04:30:39.082ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4722024-08-06T04:30:39.082ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-08-06 04:20:39.054429450 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4732024-08-06T04:30:39.082ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 28.542687ms iteration = 7 status = {"expiration":"2024-08-06T04:20:39.054429450Z","pruned":[]}
4742024-08-06T04:30:39.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4752024-08-06T04:30:39.083ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4762024-08-06T04:30:39.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4772024-08-06T04:30:39.083ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4782024-08-06T04:30:39.083ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4792024-08-06T04:30:39.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4802024-08-06T04:30:39.084ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
4812024-08-06T04:30:39.084ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
4822024-08-06T04:30:39.084ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 29.787416ms iteration = 7 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
4832024-08-06T04:30:39.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4842024-08-06T04:30:39.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4852024-08-06T04:30:39.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4862024-08-06T04:30:39.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4872024-08-06T04:30:39.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4882024-08-06T04:30:39.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 76e61613-583c-43ed-85e5-238a1f2bdaaf, lookup_type: ById(76e61613-583c-43ed-85e5-238a1f2bdaaf) } result = Ok(())
4892024-08-06T04:30:39.086ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4902024-08-06T04:30:39.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4912024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4922024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4932024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4942024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 14f1be63-b272-4578-a0ca-d8087b696597 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
4952024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4962024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-08-06 04:24:54.023094 UTC dns_group = internal generation = 1
4972024-08-06T04:30:39.087ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 33.491924ms iteration = 7 status = {"generation":1}
4982024-08-06T04:30:39.088ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.788836ms iteration = 13 status = 0
4992024-08-06T04:30:39.088ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
5002024-08-06T04:30:39.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5012024-08-06T04:30:39.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5022024-08-06T04:30:39.088ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5032024-08-06T04:30:39.088ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5042024-08-06T04:30:39.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5052024-08-06T04:30:39.095ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
5062024-08-06T04:30:39.095ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.842307ms iteration = 37 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
5072024-08-06T04:30:39.096ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-08-06T04:24:38.131091Z, time_modified: 2024-08-06T04:24:38.131091Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 4bc3cff2-7530-4dd3-9c1a-07349e4d2cbb, db_silo: Silo { identity: SiloIdentity { id: 4bc3cff2-7530-4dd3-9c1a-07349e4d2cbb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-06T04:24:54.579264Z, time_modified: 2024-08-06T04:24:54.579264Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "c3531cd92fc5a317acb397c70b7ecf7d73372088956234ae59b5002d53650457" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 4bc3cff2-7530-4dd3-9c1a-07349e4d2cbb, db_silo: Silo { identity: SiloIdentity { id: 4bc3cff2-7530-4dd3-9c1a-07349e4d2cbb, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-08-06T04:24:54.579264Z, time_modified: 2024-08-06T04:24:54.579264Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "c3531cd92fc5a317acb397c70b7ecf7d73372088956234ae59b5002d53650457" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
5082024-08-06T04:30:39.096ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 42.484829ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"4bc3cff2-7530-4dd3-9c1a-07349e4d2cbb","tls_certs":["c3531cd92fc5a317acb397c70b7ecf7d73372088956234ae59b5002d53650457"]}},"default_endpoint":{"silo_id":"4bc3cff2-7530-4dd3-9c1a-07349e4d2cbb","tls_certs":["c3531cd92fc5a317acb397c70b7ecf7d73372088956234ae59b5002d53650457"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5092024-08-06T04:30:39.100ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5102024-08-06T04:30:39.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5112024-08-06T04:30:39.107ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5122024-08-06T04:30:39.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5132024-08-06T04:30:39.108ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5142024-08-06T04:30:39.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5152024-08-06T04:30:39.110ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5162024-08-06T04:30:39.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5172024-08-06T04:30:39.111ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5182024-08-06T04:30:39.112ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5192024-08-06T04:30:39.112ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5202024-08-06T04:30:39.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5212024-08-06T04:30:39.276ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
5222024-08-06T04:30:39.276ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5232024-08-06T04:30:39.276ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "aec0ed40-a4e0-49d3-a864-3d387e9e718c", "content-length": "26", "date": "Tue, 06 Aug 2024 04:30:38 GMT"} })
5242024-08-06T04:30:39.276ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5252024-08-06T04:30:39.278ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5262024-08-06T04:30:39.289ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f zone_address = fd00:1122:3344:101::2
5272024-08-06T04:30:39.289ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5282024-08-06T04:30:39.289ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cd9b372e-145a-4635-8f2a-3e55ae0f39f9", "content-length": "26", "date": "Tue, 06 Aug 2024 04:30:38 GMT"} })
5292024-08-06T04:30:39.289ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5302024-08-06T04:30:39.291ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
5312024-08-06T04:30:39.292ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
5322024-08-06T04:30:39.292ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5332024-08-06T04:30:39.293ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "263531f6-45ed-4565-a8b5-6da6acb11bb0", "content-length": "26", "date": "Tue, 06 Aug 2024 04:30:38 GMT"} })
5342024-08-06T04:30:39.293ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5352024-08-06T04:30:39.294ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5362024-08-06T04:30:39.369ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5372024-08-06T04:30:39.375ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5382024-08-06T04:30:39.448ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5392024-08-06T04:30:39.448ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "875b6066-7ac8-4f36-9858-480cd428a1e8", "content-length": "810", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
5402024-08-06T04:30:39.448ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 245, 95, 122])), vni: Vni(7294630) }} sled = buskin
5412024-08-06T04:30:39.448ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
5422024-08-06T04:30:39.448ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
5432024-08-06T04:30:39.451ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 396.458728ms iteration = 13 status = {}
5442024-08-06T04:30:39.540ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/8d5fc184-3587-4d49-b550-4f3e2c749851/state
5452024-08-06T04:30:39.540ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5462024-08-06T04:30:39.541ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d8fec589-9dc5-491b-9ddc-3e6ab25cd7bc", "content-length": "2", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
5472024-08-06T04:30:39.543ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 488.724949ms iteration = 13 status = {}
5482024-08-06T04:30:39.543ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5492024-08-06T04:30:39.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5502024-08-06T04:30:39.544ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5512024-08-06T04:30:39.544ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1d1adbba-7055-4922-9f26-2d3a58a428d3", "content-length": "583", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
5522024-08-06T04:30:39.544ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5532024-08-06T04:30:39.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5542024-08-06T04:30:39.545ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 490.800184ms iteration = 13 status = null
5552024-08-06T04:30:39.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5562024-08-06T04:30:39.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5572024-08-06T04:30:39.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5582024-08-06T04:30:39.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5592024-08-06T04:30:39.552ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5602024-08-06T04:30:39.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5612024-08-06T04:30:39.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5622024-08-06T04:30:39.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5632024-08-06T04:30:39.556ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5642024-08-06T04:30:39.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5652024-08-06T04:30:39.558ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5662024-08-06T04:30:39.559ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "59a52a24-96f2-48d1-8d23-cf8b5286e0d1", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
5672024-08-06T04:30:39.560ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 505.992123ms iteration = 13 status = {}
5682024-08-06T04:30:39.590ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5692024-08-06T04:30:39.598ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5702024-08-06T04:30:39.599ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d7de54ad-e71e-491a-8a67-a11d087a5e96", "date": "Tue, 06 Aug 2024 04:30:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
5712024-08-06T04:30:39.599ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
5722024-08-06T04:30:39.600ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dbd7d65d-7691-4c45-9050-35587f79f70f", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
5732024-08-06T04:30:39.600ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
5742024-08-06T04:30:39.601ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
5752024-08-06T04:30:39.601ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 546.766389ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5762024-08-06T04:30:39.653ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5772024-08-06T04:30:39.653ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "614bd0c0-cee2-4256-9c40-3352c0d805f5", "date": "Tue, 06 Aug 2024 04:30:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5782024-08-06T04:30:39.653ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5792024-08-06T04:30:39.654ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "07d8cfe0-1e2b-45af-932c-4126bfcb08f2", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5802024-08-06T04:30:39.654ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5812024-08-06T04:30:39.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "930a5b07-eb39-42fd-99d7-270361d50379", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5822024-08-06T04:30:39.656ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5832024-08-06T04:30:39.658ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5842024-08-06T04:30:39.658ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 604.261307ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5852024-08-06T04:30:39.672ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5862024-08-06T04:30:39.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5872024-08-06T04:30:39.674ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: c3fd3796-62f1-488f-a5ed-505375c3218a, rack_id: ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e29905d0-96f1-4b4c-be22-f8c46aedb308) } rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0
5882024-08-06T04:30:39.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5892024-08-06T04:30:39.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5902024-08-06T04:30:39.680ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5912024-08-06T04:30:39.681ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "51b975a6-983a-4455-8c9f-6e811ef4090f", "content-length": "121", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
5922024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
5932024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e29905d0-96f1-4b4c-be22-f8c46aedb308
5942024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
5952024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
5962024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {}
5972024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {}
5982024-08-06T04:30:39.681ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
5992024-08-06T04:30:39.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6002024-08-06T04:30:39.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6012024-08-06T04:30:39.682ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6022024-08-06T04:30:39.683ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "36ac7926-d0e4-4d4e-8255-c4d73eeeda29", "content-length": "2", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
6032024-08-06T04:30:39.683ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6042024-08-06T04:30:39.683ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "68027423-3e3f-4626-b1a5-061cb4e91130", "content-length": "52", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
6052024-08-06T04:30:39.683ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0
6062024-08-06T04:30:39.683ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6072024-08-06T04:30:39.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d7600955-81ac-4a47-98ab-646ae84d22b5", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
6082024-08-06T04:30:39.730ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch = Switch0
6092024-08-06T04:30:39.730ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6102024-08-06T04:30:39.731ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1d6e88e9-41ff-418c-b9c6-8577d08d1d0e", "content-length": "494", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
6112024-08-06T04:30:39.731ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6122024-08-06T04:30:39.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6132024-08-06T04:30:39.734ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6142024-08-06T04:30:39.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6152024-08-06T04:30:39.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6162024-08-06T04:30:39.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6172024-08-06T04:30:39.737ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6182024-08-06T04:30:39.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6192024-08-06T04:30:39.744ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 689.944569ms iteration = 13 status = {}
6202024-08-06T04:30:39.807ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 437 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = 992b98a4-04f2-4e3e-87ce-8d90b62e5470 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
6212024-08-06T04:30:39.809ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
6222024-08-06T04:30:39.809ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
6232024-08-06T04:30:39.838ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
6242024-08-06T04:30:39.838ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:64083
6252024-08-06T04:30:39.839ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:64083 req_id = 10b0cd9e-df2d-4bf5-ac1a-5e478ff3f700 roles = RoleSet { roles: {} } uri = /metrics/producers
6262024-08-06T04:30:39.839ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:64083 req_id = 10b0cd9e-df2d-4bf5-ac1a-5e478ff3f700 resource = Database result = Ok(()) uri = /metrics/producers
6272024-08-06T04:30:39.870ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 73df88a2-da9b-4397-971d-68e15e9f0d2f
6282024-08-06T04:30:39.870ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:64083 req_id = 10b0cd9e-df2d-4bf5-ac1a-5e478ff3f700 roles = RoleSet { roles: {} } uri = /metrics/producers
6292024-08-06T04:30:39.871ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:64083 req_id = 10b0cd9e-df2d-4bf5-ac1a-5e478ff3f700 resource = Database result = Ok(()) uri = /metrics/producers
6302024-08-06T04:30:39.899ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 73df88a2-da9b-4397-971d-68e15e9f0d2f uri = http://[fd00:1122:3344:101::d]:12223/producers
6312024-08-06T04:30:39.900ZDEBGnexus: client response oximeter-collector = 73df88a2-da9b-4397-971d-68e15e9f0d2f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d8ebf89b-1f2d-4494-b179-391746265ba1", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
6322024-08-06T04:30:39.900ZINFOnexus: assigned collector to new producer collector_id = 73df88a2-da9b-4397-971d-68e15e9f0d2f file = nexus/src/app/oximeter.rs:125 producer_id = f7d776bb-5aec-48b2-8c5e-56a66a693ad1
6332024-08-06T04:30:39.901ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 62518 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:64083 req_id = 10b0cd9e-df2d-4bf5-ac1a-5e478ff3f700 response_code = 201 uri = /metrics/producers
6342024-08-06T04:30:39.901ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "10b0cd9e-df2d-4bf5-ac1a-5e478ff3f700", "content-length": "41", "date": "Tue, 06 Aug 2024 04:30:39 GMT"} })
6352024-08-06T04:30:39.903ZDEBGnexus (producer-registration-task): registered with nexus successfully
6362024-08-06T04:30:39.903ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
6372024-08-06T04:30:41.352ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/8d5fc184-3587-4d49-b550-4f3e2c749851/state", query: None, fragment: None }, status: 404, headers: {"content-type": "application/json", "x-request-id": "92d5a8b6-a803-4f50-9159-774680996ad1", "content-length": "120", "date": "Tue, 06 Aug 2024 04:30:41 GMT"} })
6382024-08-06T04:30:41.352ZINFOnexus (BackgroundTasks): instance is wayyyyy gone background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:102 instance_id = 8d5fc184-3587-4d49-b550-4f3e2c749851
6392024-08-06T04:30:41.356ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 1 total_failed = 1 total_incomplete = 0 total_instances = 1
6402024-08-06T04:30:41.356ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 2.302604177s iteration = 13 status = {"failed_checks":{"no_such_instance":1},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":1}
6412024-08-06T04:30:49.054ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
6422024-08-06T04:30:49.055ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
6432024-08-06T04:30:49.055ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
6442024-08-06T04:30:49.055ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6452024-08-06T04:30:49.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6462024-08-06T04:30:49.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6472024-08-06T04:30:49.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6482024-08-06T04:30:49.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6492024-08-06T04:30:49.057ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6502024-08-06T04:30:49.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6512024-08-06T04:30:49.058ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
6522024-08-06T04:30:49.058ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.928528ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6532024-08-06T04:30:49.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6542024-08-06T04:30:49.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6552024-08-06T04:30:49.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6562024-08-06T04:30:49.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6572024-08-06T04:30:49.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6582024-08-06T04:30:49.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6592024-08-06T04:30:49.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
6602024-08-06T04:30:49.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6612024-08-06T04:30:49.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6622024-08-06T04:30:49.070ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
6632024-08-06T04:30:49.070ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.978026ms iteration = 38 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
6642024-08-06T04:30:49.902ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 579 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = 3d3a28fa-aaae-4205-ba8e-5079f7c22cd4 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
6652024-08-06T04:30:59.055ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
6662024-08-06T04:30:59.055ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
6672024-08-06T04:30:59.055ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6682024-08-06T04:30:59.055ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
6692024-08-06T04:30:59.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6702024-08-06T04:30:59.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6712024-08-06T04:30:59.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6722024-08-06T04:30:59.057ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6732024-08-06T04:30:59.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6742024-08-06T04:30:59.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6752024-08-06T04:30:59.058ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
6762024-08-06T04:30:59.058ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.295034ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6772024-08-06T04:30:59.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6782024-08-06T04:30:59.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6792024-08-06T04:30:59.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6802024-08-06T04:30:59.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6812024-08-06T04:30:59.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6822024-08-06T04:30:59.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6832024-08-06T04:30:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
6842024-08-06T04:30:59.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6852024-08-06T04:30:59.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6862024-08-06T04:30:59.071ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
6872024-08-06T04:30:59.071ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.125217ms iteration = 39 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
6882024-08-06T04:30:59.902ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 656 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = fd28e104-9db9-437d-8629-05e31025afd8 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
6892024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
6902024-08-06T04:31:09.055ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
6912024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6922024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
6932024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6942024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
6952024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6962024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
6972024-08-06T04:31:09.055ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
6982024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
6992024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7002024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7012024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
7022024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
7032024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7042024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
7052024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7062024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7072024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
7082024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7092024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
7102024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
7112024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7122024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7132024-08-06T04:31:09.055ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
7142024-08-06T04:31:09.055ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
7152024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7162024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7172024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7182024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7192024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7202024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7212024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7222024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7232024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7242024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7252024-08-06T04:31:09.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7262024-08-06T04:31:09.057ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7272024-08-06T04:31:09.058ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7282024-08-06T04:31:09.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7292024-08-06T04:31:09.058ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7302024-08-06T04:31:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7312024-08-06T04:31:09.060ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7322024-08-06T04:31:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7332024-08-06T04:31:09.060ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7342024-08-06T04:31:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7352024-08-06T04:31:09.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7362024-08-06T04:31:09.062ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7372024-08-06T04:31:09.063ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1265, tv_nsec: 150670529 } })
7382024-08-06T04:31:09.063ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7392024-08-06T04:31:09.065ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
7402024-08-06T04:31:09.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7412024-08-06T04:31:09.065ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7422024-08-06T04:31:09.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7432024-08-06T04:31:09.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7442024-08-06T04:31:09.066ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
7452024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 10.726158ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7462024-08-06T04:31:09.066ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
7472024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.605797ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7482024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7492024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7502024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7512024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7522024-08-06T04:31:09.066ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
7532024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7542024-08-06T04:31:09.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7552024-08-06T04:31:09.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7562024-08-06T04:31:09.067ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1265, tv_nsec: 154471607 } })
7572024-08-06T04:31:09.067ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7582024-08-06T04:31:09.068ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-76e61613-583c-43ed-85e5-238a1f2bdaaf.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1265, tv_nsec: 155570944 } })
7592024-08-06T04:31:09.068ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
7602024-08-06T04:31:09.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7612024-08-06T04:31:09.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7622024-08-06T04:31:09.068ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7632024-08-06T04:31:09.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7642024-08-06T04:31:09.069ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
7652024-08-06T04:31:09.070ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7662024-08-06T04:31:09.070ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7672024-08-06T04:31:09.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7682024-08-06T04:31:09.071ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
7692024-08-06T04:31:09.071ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 15.916086ms iteration = 14 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
7702024-08-06T04:31:09.071ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7712024-08-06T04:31:09.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7722024-08-06T04:31:09.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7732024-08-06T04:31:09.072ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7742024-08-06T04:31:09.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7752024-08-06T04:31:09.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7762024-08-06T04:31:09.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7772024-08-06T04:31:09.073ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7782024-08-06T04:31:09.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7792024-08-06T04:31:09.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7802024-08-06T04:31:09.074ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7812024-08-06T04:31:09.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7822024-08-06T04:31:09.075ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
7832024-08-06T04:31:09.075ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 19.512302ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0}
7842024-08-06T04:31:09.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
7852024-08-06T04:31:09.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7862024-08-06T04:31:09.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7872024-08-06T04:31:09.077ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
7882024-08-06T04:31:09.077ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.345675ms iteration = 15 status = {"physical_disks_added":0}
7892024-08-06T04:31:09.080ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7902024-08-06T04:31:09.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7912024-08-06T04:31:09.082ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
7922024-08-06T04:31:09.082ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.866725ms iteration = 40 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
7932024-08-06T04:31:09.086ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7942024-08-06T04:31:09.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7952024-08-06T04:31:09.088ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7962024-08-06T04:31:09.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7972024-08-06T04:31:09.088ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7982024-08-06T04:31:09.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7992024-08-06T04:31:09.089ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8002024-08-06T04:31:09.089ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8012024-08-06T04:31:09.091ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8022024-08-06T04:31:09.091ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8032024-08-06T04:31:09.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8042024-08-06T04:31:09.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 76e61613-583c-43ed-85e5-238a1f2bdaaf, lookup_type: ById(76e61613-583c-43ed-85e5-238a1f2bdaaf) } result = Ok(())
8052024-08-06T04:31:09.092ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8062024-08-06T04:31:09.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8072024-08-06T04:31:09.092ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8082024-08-06T04:31:09.094ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 38.459509ms iteration = 14 status = 0
8092024-08-06T04:31:09.094ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8102024-08-06T04:31:09.094ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8112024-08-06T04:31:09.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8122024-08-06T04:31:09.182ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
8132024-08-06T04:31:09.182ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8142024-08-06T04:31:09.182ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0912c560-78fa-40d1-acf1-543c3d27547c", "content-length": "26", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8152024-08-06T04:31:09.182ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8162024-08-06T04:31:09.184ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8172024-08-06T04:31:09.184ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f zone_address = fd00:1122:3344:101::2
8182024-08-06T04:31:09.184ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8192024-08-06T04:31:09.185ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "408efa5c-5a47-4f2c-a704-04ad2afaf25f", "content-length": "26", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8202024-08-06T04:31:09.185ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8212024-08-06T04:31:09.187ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
8222024-08-06T04:31:09.197ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8232024-08-06T04:31:09.198ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "728674d4-3742-4ba3-b469-f1d1df7d65cd", "content-length": "372", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8242024-08-06T04:31:09.198ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8252024-08-06T04:31:09.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8262024-08-06T04:31:09.199ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8272024-08-06T04:31:09.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8282024-08-06T04:31:09.202ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8292024-08-06T04:31:09.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8302024-08-06T04:31:09.204ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8312024-08-06T04:31:09.204ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cce85761-b052-4825-8ef2-49af1754dede", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8322024-08-06T04:31:09.205ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 149.956539ms iteration = 14 status = {}
8332024-08-06T04:31:09.213ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
8342024-08-06T04:31:09.213ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8352024-08-06T04:31:09.213ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bf5cc7a6-fd00-4a83-803d-096f2973b73d", "content-length": "26", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8362024-08-06T04:31:09.213ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8372024-08-06T04:31:09.214ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8382024-08-06T04:31:09.226ZDEBGnexus (BackgroundTasks): client request SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8392024-08-06T04:31:09.227ZDEBGnexus (BackgroundTasks): client response SledAgent = 76e61613-583c-43ed-85e5-238a1f2bdaaf background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ecb5a354-ac44-4791-9a47-ba809a65fd2c", "content-length": "691", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8402024-08-06T04:31:09.227ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
8412024-08-06T04:31:09.227ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
8422024-08-06T04:31:09.227ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
8432024-08-06T04:31:09.229ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 173.703071ms iteration = 14 status = {}
8442024-08-06T04:31:09.321ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 265.629458ms iteration = 14 status = {}
8452024-08-06T04:31:09.374ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8462024-08-06T04:31:09.374ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "30c7adfb-1397-45d6-a9a1-09a33f86ddf6", "content-length": "2", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8472024-08-06T04:31:09.376ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8482024-08-06T04:31:09.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8492024-08-06T04:31:09.378ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 322.498322ms iteration = 14 status = null
8502024-08-06T04:31:09.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8512024-08-06T04:31:09.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8522024-08-06T04:31:09.437ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: c3fd3796-62f1-488f-a5ed-505375c3218a, rack_id: ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(e29905d0-96f1-4b4c-be22-f8c46aedb308) } rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0
8532024-08-06T04:31:09.437ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8542024-08-06T04:31:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8552024-08-06T04:31:09.444ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8562024-08-06T04:31:09.445ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "75ccd9fd-60e2-4b10-badf-b82db1acd3fd", "content-length": "121", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8572024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
8582024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = e29905d0-96f1-4b4c-be22-f8c46aedb308
8592024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
8602024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
8612024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {}
8622024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f routes = {}
8632024-08-06T04:31:09.445ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f
8642024-08-06T04:31:09.445ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8652024-08-06T04:31:09.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8662024-08-06T04:31:09.447ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8672024-08-06T04:31:09.447ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8b37e78b-84ed-4be3-9faa-8e7792baeda1", "content-length": "2", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8682024-08-06T04:31:09.447ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8692024-08-06T04:31:09.447ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7f3bdf16-2938-4d77-8b56-fa986c4bb51b", "content-length": "52", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8702024-08-06T04:31:09.447ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch_location = Switch0
8712024-08-06T04:31:09.447ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8722024-08-06T04:31:09.493ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2969401f-3d79-48d3-af37-f99f37cf194a", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8732024-08-06T04:31:09.493ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f switch = Switch0
8742024-08-06T04:31:09.493ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8752024-08-06T04:31:09.497ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ef6d27bf-b0ba-49e0-adce-e3bfb5c7060f result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "82be85e1-f6df-47f5-9875-e0e544e2d288", "content-length": "494", "date": "Tue, 06 Aug 2024 04:31:08 GMT"} })
8762024-08-06T04:31:09.497ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8772024-08-06T04:31:09.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8782024-08-06T04:31:09.500ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8792024-08-06T04:31:09.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8802024-08-06T04:31:09.502ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8812024-08-06T04:31:09.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8822024-08-06T04:31:09.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8832024-08-06T04:31:09.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8842024-08-06T04:31:09.510ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 454.238748ms iteration = 14 status = {}
8852024-08-06T04:31:09.901ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 396 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = a0df5946-0d69-48be-bb3a-c252289ca207 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
8862024-08-06T04:31:19.055ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
8872024-08-06T04:31:19.055ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
8882024-08-06T04:31:19.055ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8892024-08-06T04:31:19.055ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
8902024-08-06T04:31:19.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8912024-08-06T04:31:19.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8922024-08-06T04:31:19.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8932024-08-06T04:31:19.057ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8942024-08-06T04:31:19.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8952024-08-06T04:31:19.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8962024-08-06T04:31:19.058ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
8972024-08-06T04:31:19.058ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.673347ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8982024-08-06T04:31:19.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8992024-08-06T04:31:19.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9002024-08-06T04:31:19.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9012024-08-06T04:31:19.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9022024-08-06T04:31:19.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9032024-08-06T04:31:19.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042024-08-06T04:31:19.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
9052024-08-06T04:31:19.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9062024-08-06T04:31:19.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9072024-08-06T04:31:19.071ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
9082024-08-06T04:31:19.071ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.636951ms iteration = 41 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
9092024-08-06T04:31:19.903ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1082 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = bfc9974f-c3db-4194-a0fe-93766d922761 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1
9102024-08-06T04:31:29.055ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
9112024-08-06T04:31:29.055ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
9122024-08-06T04:31:29.055ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9132024-08-06T04:31:29.055ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
9142024-08-06T04:31:29.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9152024-08-06T04:31:29.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9162024-08-06T04:31:29.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9172024-08-06T04:31:29.057ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9182024-08-06T04:31:29.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9192024-08-06T04:31:29.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9202024-08-06T04:31:29.059ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
9212024-08-06T04:31:29.059ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.692307ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9222024-08-06T04:31:29.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9232024-08-06T04:31:29.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9242024-08-06T04:31:29.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9252024-08-06T04:31:29.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9262024-08-06T04:31:29.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9272024-08-06T04:31:29.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9282024-08-06T04:31:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: d234c62d-4359-4882-b5c9-00f71d500cb3, lookup_type: ById(d234c62d-4359-4882-b5c9-00f71d500cb3) } result = Ok(())
9292024-08-06T04:31:29.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9302024-08-06T04:31:29.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9312024-08-06T04:31:29.071ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 original_time_created = 2024-08-06 04:24:27.251480 UTC target_id = d234c62d-4359-4882-b5c9-00f71d500cb3 time_created = 2024-08-06 04:24:27.251480 UTC
9322024-08-06T04:31:29.071ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.5709ms iteration = 42 status = {"status":"target blueprint unchanged","target_id":"d234c62d-4359-4882-b5c9-00f71d500cb3","time_created":"2024-08-06T04:24:27.251480Z"}
9332024-08-06T04:31:29.902ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/52d900a/dropshot/src/server.rs:914 latency_us = 1207 local_addr = [fd00:1122:3344:101::c]:61366 method = GET remote_addr = [fd00:1122:3344:101::d]:47085 req_id = 488f5993-d1b0-40a3-a91b-080fecc663e0 response_code = 200 uri = /f7d776bb-5aec-48b2-8c5e-56a66a693ad1