Max level shown:
12024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
22024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
42024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
52024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
62024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
72024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
82024-05-10T21:30:08.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
92024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102024-05-10T21:30:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
112024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
122024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
132024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
142024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
152024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
172024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
182024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
192024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
202024-05-10T21:30:08.789ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
212024-05-10T21:30:08.789ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
222024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.35\u{b5}s iteration = 12
232024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
242024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
252024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
262024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
272024-05-10T21:30:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
282024-05-10T21:30:08.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
292024-05-10T21:30:08.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
302024-05-10T21:30:08.791ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
312024-05-10T21:30:08.791ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.523343ms iteration = 12
322024-05-10T21:30:08.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
332024-05-10T21:30:08.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
342024-05-10T21:30:08.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
352024-05-10T21:30:08.792ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1699, tv_nsec: 569405437 } })
362024-05-10T21:30:08.792ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1699, tv_nsec: 569527220 } })
372024-05-10T21:30:08.792ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
382024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
392024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
402024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
412024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
422024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
432024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
442024-05-10T21:30:08.795ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
452024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
462024-05-10T21:30:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
472024-05-10T21:30:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
482024-05-10T21:30:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
492024-05-10T21:30:08.796ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
502024-05-10T21:30:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
512024-05-10T21:30:08.796ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
522024-05-10T21:30:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
532024-05-10T21:30:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
542024-05-10T21:30:08.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
552024-05-10T21:30:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
562024-05-10T21:30:08.798ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1699, tv_nsec: 575549076 } })
572024-05-10T21:30:08.798ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
582024-05-10T21:30:08.798ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
592024-05-10T21:30:08.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
602024-05-10T21:30:08.798ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
612024-05-10T21:30:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
622024-05-10T21:30:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
632024-05-10T21:30:08.798ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
642024-05-10T21:30:08.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
652024-05-10T21:30:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
662024-05-10T21:30:08.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
672024-05-10T21:30:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
682024-05-10T21:30:08.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
692024-05-10T21:30:08.803ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
702024-05-10T21:30:08.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
712024-05-10T21:30:08.807ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
722024-05-10T21:30:08.807ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.224213ms iteration = 34
732024-05-10T21:30:08.810ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
742024-05-10T21:30:08.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
752024-05-10T21:30:08.812ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
762024-05-10T21:30:08.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
772024-05-10T21:30:08.813ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
782024-05-10T21:30:08.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
792024-05-10T21:30:08.816ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
802024-05-10T21:30:08.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
812024-05-10T21:30:08.818ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.508928ms iteration = 12
822024-05-10T21:30:08.850ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
832024-05-10T21:30:08.850ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 61.236934ms iteration = 12
842024-05-10T21:30:08.872ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
852024-05-10T21:30:08.872ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
862024-05-10T21:30:08.873ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fe7dd22c-bdc4-4f0c-9570-925d2b4cf90e", "content-length": "26", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
872024-05-10T21:30:08.873ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
882024-05-10T21:30:08.874ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
892024-05-10T21:30:08.874ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 84.643635ms iteration = 13
902024-05-10T21:30:08.875ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
912024-05-10T21:30:08.977ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
922024-05-10T21:30:08.978ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bdc8170a-d978-4632-a9d2-ffb8662ae044", "content-length": "1", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
932024-05-10T21:30:08.980ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
942024-05-10T21:30:08.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
952024-05-10T21:30:08.988ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 200.124805ms iteration = 12
962024-05-10T21:30:08.996ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
972024-05-10T21:30:08.996ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
982024-05-10T21:30:08.996ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c46b85b0-c018-45a9-ac42-1eee4cc76ba5", "content-length": "26", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
992024-05-10T21:30:08.996ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1002024-05-10T21:30:08.997ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
1012024-05-10T21:30:08.997ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1022024-05-10T21:30:08.997ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "74fcc112-03b8-4580-b5c7-c3f6e051c10f", "content-length": "26", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1032024-05-10T21:30:08.997ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1042024-05-10T21:30:08.998ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1052024-05-10T21:30:08.998ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
1062024-05-10T21:30:09.036ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 248.176013ms iteration = 12
1072024-05-10T21:30:09.094ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1082024-05-10T21:30:09.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1092024-05-10T21:30:09.096ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1102024-05-10T21:30:09.096ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1112024-05-10T21:30:09.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1122024-05-10T21:30:09.105ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1132024-05-10T21:30:09.106ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "959d7ab2-997f-46a5-8ecb-0b486ccb79eb", "content-length": "121", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1142024-05-10T21:30:09.106ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
1152024-05-10T21:30:09.106ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
1162024-05-10T21:30:09.107ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1172024-05-10T21:30:09.107ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
1182024-05-10T21:30:09.107ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
1192024-05-10T21:30:09.107ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1202024-05-10T21:30:09.107ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
1212024-05-10T21:30:09.107ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
1222024-05-10T21:30:09.107ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
1232024-05-10T21:30:09.107ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1242024-05-10T21:30:09.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1252024-05-10T21:30:09.108ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1262024-05-10T21:30:09.108ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9cc56e02-73d4-4e74-987a-7c635cd116e0", "content-length": "2", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1272024-05-10T21:30:09.108ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1282024-05-10T21:30:09.108ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "02695b7e-5070-4381-baba-f5630bd486a5", "content-length": "51", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1292024-05-10T21:30:09.108ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1302024-05-10T21:30:09.108ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1312024-05-10T21:30:09.148ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c9ba91e5-727b-49e6-a5b1-fb91045cc014", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1322024-05-10T21:30:09.148ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
1332024-05-10T21:30:09.148ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1342024-05-10T21:30:09.149ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "35fe319f-339f-40db-9567-0a1fbe66d72e", "content-length": "467", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1352024-05-10T21:30:09.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1362024-05-10T21:30:09.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1372024-05-10T21:30:09.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1382024-05-10T21:30:09.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1392024-05-10T21:30:09.155ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1402024-05-10T21:30:09.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1412024-05-10T21:30:09.157ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1422024-05-10T21:30:09.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1432024-05-10T21:30:09.163ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 374.101532ms iteration = 12
1442024-05-10T21:30:09.430ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 87 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 869c362a-16b2-45b4-a1f9-632f7d0021a6 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
1452024-05-10T21:30:18.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
1462024-05-10T21:30:18.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1472024-05-10T21:30:18.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1482024-05-10T21:30:18.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1492024-05-10T21:30:18.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1502024-05-10T21:30:18.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1512024-05-10T21:30:18.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1522024-05-10T21:30:18.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1532024-05-10T21:30:18.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1542024-05-10T21:30:18.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1552024-05-10T21:30:18.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
1562024-05-10T21:30:18.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1572024-05-10T21:30:18.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1582024-05-10T21:30:18.804ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
1592024-05-10T21:30:18.804ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.034397ms iteration = 35
1602024-05-10T21:30:19.430ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 99 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = b15ace91-a1a7-4705-ac2a-b07235b2f3db response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
1612024-05-10T21:30:28.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
1622024-05-10T21:30:28.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1632024-05-10T21:30:28.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1642024-05-10T21:30:28.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1652024-05-10T21:30:28.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1662024-05-10T21:30:28.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1672024-05-10T21:30:28.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1682024-05-10T21:30:28.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1692024-05-10T21:30:28.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1702024-05-10T21:30:28.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1712024-05-10T21:30:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
1722024-05-10T21:30:28.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1732024-05-10T21:30:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1742024-05-10T21:30:28.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
1752024-05-10T21:30:28.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.75062ms iteration = 36
1762024-05-10T21:30:29.430ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 100 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = c3acaf82-629d-43ce-936d-5d378095783f response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
1772024-05-10T21:30:38.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
1782024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
1792024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1802024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1812024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1822024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
1832024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
1842024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
1852024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1862024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1872024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
1882024-05-10T21:30:38.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
1892024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
1902024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
1912024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
1922024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1932024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
1942024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
1952024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
1962024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
1972024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1982024-05-10T21:30:38.788ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
1992024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.09\u{b5}s iteration = 7
2002024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
2012024-05-10T21:30:38.788ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
2022024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.35\u{b5}s iteration = 8
2032024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2042024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2052024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2062024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2072024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2082024-05-10T21:30:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2092024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2102024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
2112024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
2122024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2132024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
2142024-05-10T21:30:38.789ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
2152024-05-10T21:30:38.789ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
2162024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2172024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.99\u{b5}s iteration = 13
2182024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
2192024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
2202024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2212024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2222024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2232024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2242024-05-10T21:30:38.789ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 567162348 } })
2252024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2262024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2272024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2282024-05-10T21:30:38.789ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 567234610 } })
2292024-05-10T21:30:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2302024-05-10T21:30:38.790ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2312024-05-10T21:30:38.790ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2322024-05-10T21:30:38.790ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.622745ms iteration = 13
2332024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2352024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
2362024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.304339ms iteration = 7
2372024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2382024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2392024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2402024-05-10T21:30:38.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2412024-05-10T21:30:38.792ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
2422024-05-10T21:30:38.792ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.601987ms iteration = 7
2432024-05-10T21:30:38.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2442024-05-10T21:30:38.793ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 571105001 } })
2452024-05-10T21:30:38.793ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 571297305 } })
2462024-05-10T21:30:38.794ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2472024-05-10T21:30:38.794ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
2482024-05-10T21:30:38.794ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2492024-05-10T21:30:38.794ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
2502024-05-10T21:30:38.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2512024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2522024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2532024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2542024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2552024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2562024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2572024-05-10T21:30:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2582024-05-10T21:30:38.797ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2592024-05-10T21:30:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2602024-05-10T21:30:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2612024-05-10T21:30:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2622024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2632024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2642024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2652024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2662024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2672024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2682024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2692024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2702024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2712024-05-10T21:30:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2722024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2732024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2742024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2752024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2762024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2772024-05-10T21:30:38.799ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
2782024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2792024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2802024-05-10T21:30:38.799ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2812024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2822024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2832024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2842024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2852024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2862024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2872024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2882024-05-10T21:30:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2892024-05-10T21:30:38.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2902024-05-10T21:30:38.801ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2912024-05-10T21:30:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2922024-05-10T21:30:38.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2932024-05-10T21:30:38.803ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 580573190 } })
2942024-05-10T21:30:38.803ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
2952024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2962024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2972024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2982024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2992024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3002024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3012024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3022024-05-10T21:30:38.804ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3032024-05-10T21:30:38.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3042024-05-10T21:30:38.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
3052024-05-10T21:30:38.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3062024-05-10T21:30:38.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3072024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3082024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3092024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3102024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
3112024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3122024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3132024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3142024-05-10T21:30:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3152024-05-10T21:30:38.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3162024-05-10T21:30:38.807ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3172024-05-10T21:30:38.807ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
3182024-05-10T21:30:38.807ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 18.517359ms iteration = 14
3192024-05-10T21:30:38.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3202024-05-10T21:30:38.808ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3212024-05-10T21:30:38.808ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3222024-05-10T21:30:38.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3232024-05-10T21:30:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3242024-05-10T21:30:38.810ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3252024-05-10T21:30:38.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3262024-05-10T21:30:38.810ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3272024-05-10T21:30:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3282024-05-10T21:30:38.810ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
3292024-05-10T21:30:38.810ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.948741ms iteration = 37
3302024-05-10T21:30:38.811ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3312024-05-10T21:30:38.812ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3322024-05-10T21:30:38.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3332024-05-10T21:30:38.813ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
3342024-05-10T21:30:38.813ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3352024-05-10T21:30:38.813ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
3362024-05-10T21:30:38.813ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.688408ms iteration = 7
3372024-05-10T21:30:38.813ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3382024-05-10T21:30:38.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3392024-05-10T21:30:38.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3402024-05-10T21:30:38.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3412024-05-10T21:30:38.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3422024-05-10T21:30:38.815ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
3432024-05-10T21:30:38.815ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3442024-05-10T21:30:38.815ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
3452024-05-10T21:30:38.815ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.899774ms iteration = 7
3462024-05-10T21:30:38.815ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3472024-05-10T21:30:38.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3482024-05-10T21:30:38.817ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3492024-05-10T21:30:38.820ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
3502024-05-10T21:30:38.820ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3512024-05-10T21:30:38.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3522024-05-10T21:30:38.821ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.849537ms iteration = 13
3532024-05-10T21:30:38.825ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
3542024-05-10T21:30:38.825ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 37.312392ms iteration = 7
3552024-05-10T21:30:38.941ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3562024-05-10T21:30:38.980ZDEBGnexus (BackgroundTasks): client request SledAgent = 3ae06dcf-597c-4d18-b2d7-37c1f02096d7 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/ad3c3a26-88d8-4e2a-8352-8fd346279818/state
3572024-05-10T21:30:38.982ZDEBGnexus (BackgroundTasks): client response SledAgent = 3ae06dcf-597c-4d18-b2d7-37c1f02096d7 background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/ad3c3a26-88d8-4e2a-8352-8fd346279818/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a5a7e60-c11b-43d7-9ce7-ae3bc2a3abd4", "content-length": "313", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3582024-05-10T21:30:38.988ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 state = Running
3592024-05-10T21:30:38.990ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
3602024-05-10T21:30:38.990ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3612024-05-10T21:30:38.990ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fff274f0-9a95-4589-b441-6ab4d409f9fa", "content-length": "26", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3622024-05-10T21:30:38.990ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3632024-05-10T21:30:38.990ZINFOnexus (BackgroundTasks): received new runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1979 instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 instance_state = InstanceRuntimeState { propolis_id: Some(6182ec66-497a-4e01-b9cc-9718ced2da39), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-05-10T21:30:21.845829526Z } propolis_id = 6182ec66-497a-4e01-b9cc-9718ced2da39 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-05-10T21:30:29.534835444Z }
3642024-05-10T21:30:38.991ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
3652024-05-10T21:30:38.991ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3662024-05-10T21:30:38.991ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70f49dca-f3e7-4d38-b51a-59619b566bbc", "content-length": "26", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3672024-05-10T21:30:38.991ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3682024-05-10T21:30:38.992ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
3692024-05-10T21:30:38.993ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3702024-05-10T21:30:38.993ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3712024-05-10T21:30:38.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3722024-05-10T21:30:38.997ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3732024-05-10T21:30:38.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3742024-05-10T21:30:38.999ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3752024-05-10T21:30:38.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3762024-05-10T21:30:39.000ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3772024-05-10T21:30:39.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3782024-05-10T21:30:39.001ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3792024-05-10T21:30:39.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3802024-05-10T21:30:39.002ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3812024-05-10T21:30:39.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3822024-05-10T21:30:39.004ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3832024-05-10T21:30:39.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 47e71290-fa42-43c0-acb1-5a418035ba16, lookup_type: ById(47e71290-fa42-43c0-acb1-5a418035ba16) }, key: 45ce0783-2703-487d-910e-d5b22706e48e, lookup_type: ById(45ce0783-2703-487d-910e-d5b22706e48e) }, key: ad3c3a26-88d8-4e2a-8352-8fd346279818, lookup_type: ById(ad3c3a26-88d8-4e2a-8352-8fd346279818) } result = Ok(())
3842024-05-10T21:30:39.007ZDEBGnexus (BackgroundTasks): instance state generation already advanced, won't touch network config background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818
3852024-05-10T21:30:39.044ZINFOnexus (BackgroundTasks): instance and vmm updated by sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2112 instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 instance_updated = false propolis_id = 6182ec66-497a-4e01-b9cc-9718ced2da39 vmm_updated = false
3862024-05-10T21:30:39.044ZDEBGnexus (BackgroundTasks): update successful background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 instance_updated = false state = Running vmm_updated = false
3872024-05-10T21:30:39.047ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1
3882024-05-10T21:30:39.047ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 257.722562ms iteration = 13
3892024-05-10T21:30:39.059ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3902024-05-10T21:30:39.106ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3912024-05-10T21:30:39.107ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d84f46df-c431-41b0-b666-1f6d1edeb6ef", "date": "Fri, 10 May 2024 21:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3922024-05-10T21:30:39.107ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3932024-05-10T21:30:39.108ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
3942024-05-10T21:30:39.108ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3952024-05-10T21:30:39.108ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6b846fb8-d61d-46e4-b348-b910ec8d6ef8", "content-length": "26", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3962024-05-10T21:30:39.108ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3972024-05-10T21:30:39.109ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b025322d-d375-4189-af8f-5ebae95ec63a", "date": "Fri, 10 May 2024 21:30:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3982024-05-10T21:30:39.109ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3992024-05-10T21:30:39.111ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4002024-05-10T21:30:39.112ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4012024-05-10T21:30:39.112ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 324.41468ms iteration = 9
4022024-05-10T21:30:39.211ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4032024-05-10T21:30:39.235ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4042024-05-10T21:30:39.236ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "29c2dc15-3bc6-4875-a4d4-baa5841fef4f", "content-length": "1", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4052024-05-10T21:30:39.237ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4062024-05-10T21:30:39.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4072024-05-10T21:30:39.238ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 450.901683ms iteration = 13
4082024-05-10T21:30:39.265ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 477.867847ms iteration = 13
4092024-05-10T21:30:39.297ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4102024-05-10T21:30:39.297ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ac92205a-d1c4-4f60-bb0c-31a208052a8c", "date": "Fri, 10 May 2024 21:30:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4112024-05-10T21:30:39.297ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4122024-05-10T21:30:39.299ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ff228219-c1d1-48ff-820b-ff7a6b5e627b", "date": "Fri, 10 May 2024 21:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4132024-05-10T21:30:39.299ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4142024-05-10T21:30:39.301ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2a96a61d-3eba-49c5-bc10-53290c984feb", "date": "Fri, 10 May 2024 21:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4152024-05-10T21:30:39.301ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4162024-05-10T21:30:39.303ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4172024-05-10T21:30:39.303ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 515.65451ms iteration = 9
4182024-05-10T21:30:39.340ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4192024-05-10T21:30:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4202024-05-10T21:30:39.342ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4212024-05-10T21:30:39.342ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4222024-05-10T21:30:39.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4232024-05-10T21:30:39.349ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4242024-05-10T21:30:39.350ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "076d988c-147d-4538-a3b7-0562e661ed6a", "content-length": "121", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4252024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4262024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
4272024-05-10T21:30:39.350ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4282024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
4292024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
4302024-05-10T21:30:39.350ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4312024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
4322024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
4332024-05-10T21:30:39.350ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
4342024-05-10T21:30:39.350ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4352024-05-10T21:30:39.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4362024-05-10T21:30:39.352ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4372024-05-10T21:30:39.352ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c6b1aafd-fb6c-4dc7-90fe-8b6cd9ab66db", "content-length": "2", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4382024-05-10T21:30:39.352ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4392024-05-10T21:30:39.352ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "75096eb1-2721-4f3d-acd6-18e2a11eef5a", "content-length": "51", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4402024-05-10T21:30:39.352ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4412024-05-10T21:30:39.352ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4422024-05-10T21:30:39.396ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "31785b14-3d5b-432d-ba58-39f5320b6017", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4432024-05-10T21:30:39.396ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
4442024-05-10T21:30:39.396ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4452024-05-10T21:30:39.396ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2d8ec14e-4f99-42bc-bfe0-6d9729fd88b1", "content-length": "467", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4462024-05-10T21:30:39.396ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4472024-05-10T21:30:39.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4482024-05-10T21:30:39.399ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4492024-05-10T21:30:39.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4502024-05-10T21:30:39.401ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4512024-05-10T21:30:39.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4522024-05-10T21:30:39.403ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4532024-05-10T21:30:39.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4542024-05-10T21:30:39.408ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 619.651619ms iteration = 13
4552024-05-10T21:30:39.431ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 145 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 2650b167-52e2-4379-aadc-169fb4353b64 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
4562024-05-10T21:30:39.434ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
4572024-05-10T21:30:39.434ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
4582024-05-10T21:30:39.464ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
4592024-05-10T21:30:39.465ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:46031
4602024-05-10T21:30:39.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46031 req_id = 42c6ccd0-e6e1-433c-a210-787f8befb5c9 roles = RoleSet { roles: {} } uri = /metrics/producers
4612024-05-10T21:30:39.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46031 req_id = 42c6ccd0-e6e1-433c-a210-787f8befb5c9 resource = Database result = Ok(()) uri = /metrics/producers
4622024-05-10T21:30:39.496ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 95738557-5fbe-4760-a2f5-e295833eb800
4632024-05-10T21:30:39.497ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46031 req_id = 42c6ccd0-e6e1-433c-a210-787f8befb5c9 roles = RoleSet { roles: {} } uri = /metrics/producers
4642024-05-10T21:30:39.497ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46031 req_id = 42c6ccd0-e6e1-433c-a210-787f8befb5c9 resource = Database result = Ok(()) uri = /metrics/producers
4652024-05-10T21:30:39.523ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 uri = http://[fd00:1122:3344:101::d]:12223/producers
4662024-05-10T21:30:39.523ZDEBGnexus: client response oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2ea83601-8fc8-4b31-9f6a-e8fd8dfcd104", "date": "Fri, 10 May 2024 21:30:39 GMT"} })
4672024-05-10T21:30:39.523ZINFOnexus: assigned collector to new producer collector_id = 95738557-5fbe-4760-a2f5-e295833eb800 file = nexus/src/app/oximeter.rs:125 producer_id = 7bb2217a-5bab-47ca-8e9e-29f166894592
4682024-05-10T21:30:39.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 61039 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46031 req_id = 42c6ccd0-e6e1-433c-a210-787f8befb5c9 response_code = 201 uri = /metrics/producers
4692024-05-10T21:30:39.526ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "42c6ccd0-e6e1-433c-a210-787f8befb5c9", "content-length": "41", "date": "Fri, 10 May 2024 21:30:39 GMT"} })
4702024-05-10T21:30:39.527ZDEBGnexus (producer-registration-task): registered with nexus successfully
4712024-05-10T21:30:39.527ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
4722024-05-10T21:30:48.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
4732024-05-10T21:30:48.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4742024-05-10T21:30:48.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4752024-05-10T21:30:48.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4762024-05-10T21:30:48.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4772024-05-10T21:30:48.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4782024-05-10T21:30:48.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4792024-05-10T21:30:48.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4802024-05-10T21:30:48.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4812024-05-10T21:30:48.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4822024-05-10T21:30:48.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
4832024-05-10T21:30:48.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4842024-05-10T21:30:48.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4852024-05-10T21:30:48.802ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
4862024-05-10T21:30:48.802ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.179508ms iteration = 38
4872024-05-10T21:30:49.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 776 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 2bfa1725-8cde-47e9-b581-2eb9007c01a7 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
4882024-05-10T21:30:58.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
4892024-05-10T21:30:58.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4902024-05-10T21:30:58.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4912024-05-10T21:30:58.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4922024-05-10T21:30:58.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4932024-05-10T21:30:58.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4942024-05-10T21:30:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4952024-05-10T21:30:58.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4962024-05-10T21:30:58.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4972024-05-10T21:30:58.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4982024-05-10T21:30:58.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
4992024-05-10T21:30:58.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5002024-05-10T21:30:58.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5012024-05-10T21:30:58.805ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
5022024-05-10T21:30:58.805ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.984086ms iteration = 39
5032024-05-10T21:30:59.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 161 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = be4abf35-8f21-4e6a-a98c-0214aee692dd response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
5042024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
5052024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
5062024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5072024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5082024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
5092024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5102024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
5112024-05-10T21:31:08.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5122024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
5132024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
5142024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5152024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
5162024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5172024-05-10T21:31:08.788ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5182024-05-10T21:31:08.788ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5192024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.7\u{b5}s iteration = 14
5202024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
5212024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
5222024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5232024-05-10T21:31:08.788ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5242024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5252024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5262024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5272024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5282024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5292024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5302024-05-10T21:31:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5312024-05-10T21:31:08.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5322024-05-10T21:31:08.790ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5332024-05-10T21:31:08.790ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5342024-05-10T21:31:08.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5352024-05-10T21:31:08.791ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
5362024-05-10T21:31:08.791ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.87402ms iteration = 14
5372024-05-10T21:31:08.791ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1759, tv_nsec: 569099185 } })
5382024-05-10T21:31:08.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5392024-05-10T21:31:08.792ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
5402024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5412024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5422024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5432024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5442024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5452024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5462024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5472024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5482024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5492024-05-10T21:31:08.796ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1759, tv_nsec: 574068699 } })
5502024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5512024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5522024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5532024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5542024-05-10T21:31:08.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5552024-05-10T21:31:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5562024-05-10T21:31:08.797ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
5572024-05-10T21:31:08.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5582024-05-10T21:31:08.798ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5592024-05-10T21:31:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5602024-05-10T21:31:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5612024-05-10T21:31:08.798ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
5622024-05-10T21:31:08.798ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5632024-05-10T21:31:08.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5642024-05-10T21:31:08.799ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1759, tv_nsec: 576714165 } })
5652024-05-10T21:31:08.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5662024-05-10T21:31:08.800ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5672024-05-10T21:31:08.800ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
5682024-05-10T21:31:08.800ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
5692024-05-10T21:31:08.800ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 11.9044ms iteration = 14
5702024-05-10T21:31:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5712024-05-10T21:31:08.802ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5722024-05-10T21:31:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5732024-05-10T21:31:08.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
5742024-05-10T21:31:08.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5752024-05-10T21:31:08.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5762024-05-10T21:31:08.805ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
5772024-05-10T21:31:08.805ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.876864ms iteration = 15
5782024-05-10T21:31:08.810ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
5792024-05-10T21:31:08.810ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.843968ms iteration = 40
5802024-05-10T21:31:08.815ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5812024-05-10T21:31:08.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5822024-05-10T21:31:08.816ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5832024-05-10T21:31:08.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5842024-05-10T21:31:08.817ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5852024-05-10T21:31:08.820ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
5862024-05-10T21:31:08.820ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5872024-05-10T21:31:08.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5882024-05-10T21:31:08.821ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.071393ms iteration = 14
5892024-05-10T21:31:08.900ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
5902024-05-10T21:31:08.900ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5912024-05-10T21:31:08.900ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f9a26b47-38c0-485d-82cb-d70749c85549", "content-length": "26", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
5922024-05-10T21:31:08.900ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5932024-05-10T21:31:08.903ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5942024-05-10T21:31:08.985ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
5952024-05-10T21:31:08.985ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5962024-05-10T21:31:08.986ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "acbb07fa-71cf-47e8-a3d0-eab5a2230bfe", "content-length": "26", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
5972024-05-10T21:31:08.986ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5982024-05-10T21:31:08.988ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
5992024-05-10T21:31:08.988ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6002024-05-10T21:31:08.988ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 199.604934ms iteration = 14
6012024-05-10T21:31:08.988ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bc3988e9-6a2e-434f-9e91-7cca670d6716", "content-length": "26", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6022024-05-10T21:31:08.988ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6032024-05-10T21:31:08.988ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
6042024-05-10T21:31:08.989ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6052024-05-10T21:31:09.051ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6062024-05-10T21:31:09.051ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e2ef2db3-172d-4408-8dce-e30545ccce49", "content-length": "2", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6072024-05-10T21:31:09.053ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6082024-05-10T21:31:09.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6092024-05-10T21:31:09.054ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 266.094508ms iteration = 14
6102024-05-10T21:31:09.111ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6112024-05-10T21:31:09.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6122024-05-10T21:31:09.113ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6132024-05-10T21:31:09.113ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6142024-05-10T21:31:09.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6152024-05-10T21:31:09.119ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6162024-05-10T21:31:09.120ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "39180164-9e43-490f-b2d8-fb7e2b39e9f1", "content-length": "121", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6172024-05-10T21:31:09.120ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
6182024-05-10T21:31:09.120ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
6192024-05-10T21:31:09.121ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6202024-05-10T21:31:09.121ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
6212024-05-10T21:31:09.121ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
6222024-05-10T21:31:09.121ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6232024-05-10T21:31:09.121ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
6242024-05-10T21:31:09.121ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
6252024-05-10T21:31:09.121ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
6262024-05-10T21:31:09.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6272024-05-10T21:31:09.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6282024-05-10T21:31:09.122ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6292024-05-10T21:31:09.122ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "db00e0f7-8711-4458-a74e-408e310fcd0c", "content-length": "2", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6302024-05-10T21:31:09.122ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6312024-05-10T21:31:09.122ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4efbacab-a952-4d27-a478-38ba45c9c0dc", "content-length": "51", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6322024-05-10T21:31:09.122ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6332024-05-10T21:31:09.122ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6342024-05-10T21:31:09.169ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a29a3e41-8440-48d1-bd9f-a0adbda54402", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6352024-05-10T21:31:09.169ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
6362024-05-10T21:31:09.169ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6372024-05-10T21:31:09.170ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5740d62c-5619-42b4-949d-2b41097932b5", "content-length": "467", "date": "Fri, 10 May 2024 21:31:09 GMT"} })
6382024-05-10T21:31:09.170ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6392024-05-10T21:31:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6402024-05-10T21:31:09.173ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6412024-05-10T21:31:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6422024-05-10T21:31:09.176ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6432024-05-10T21:31:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6442024-05-10T21:31:09.177ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6452024-05-10T21:31:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6462024-05-10T21:31:09.182ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 393.612802ms iteration = 14
6472024-05-10T21:31:09.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 128 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = c9a024cc-e5ea-4fd0-a5b7-8be0a0f49e78 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
6482024-05-10T21:31:18.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
6492024-05-10T21:31:18.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6502024-05-10T21:31:18.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6512024-05-10T21:31:18.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6522024-05-10T21:31:18.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6532024-05-10T21:31:18.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6542024-05-10T21:31:18.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6552024-05-10T21:31:18.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6562024-05-10T21:31:18.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6572024-05-10T21:31:18.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6582024-05-10T21:31:18.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
6592024-05-10T21:31:18.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6602024-05-10T21:31:18.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6612024-05-10T21:31:18.802ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
6622024-05-10T21:31:18.802ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.977304ms iteration = 41
6632024-05-10T21:31:19.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 831 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 16af9d61-c84a-45a7-8709-6b0111a62afd response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
6642024-05-10T21:31:28.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
6652024-05-10T21:31:28.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6662024-05-10T21:31:28.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6672024-05-10T21:31:28.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6682024-05-10T21:31:28.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6692024-05-10T21:31:28.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6702024-05-10T21:31:28.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6712024-05-10T21:31:28.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6722024-05-10T21:31:28.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6732024-05-10T21:31:28.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6742024-05-10T21:31:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
6752024-05-10T21:31:28.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6762024-05-10T21:31:28.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6772024-05-10T21:31:28.804ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
6782024-05-10T21:31:28.804ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.169699ms iteration = 42
6792024-05-10T21:31:29.524ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 96 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 6166cb70-0310-4bed-af71-edc1c27c5206 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
6802024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
6812024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
6822024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6832024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
6842024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6852024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6862024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
6872024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
6882024-05-10T21:31:38.788ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
6892024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.66\u{b5}s iteration = 8
6902024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
6912024-05-10T21:31:38.788ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
6922024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
6932024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.67\u{b5}s iteration = 9
6942024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
6952024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6962024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6972024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
6982024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
6992024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
7002024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7012024-05-10T21:31:38.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7022024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7032024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
7042024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
7052024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7062024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7072024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7082024-05-10T21:31:38.788ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 566048368 } })
7092024-05-10T21:31:38.788ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 566060768 } })
7102024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7112024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7122024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
7132024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
7142024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
7152024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7162024-05-10T21:31:38.788ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
7172024-05-10T21:31:38.788ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
7182024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
7192024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7202024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.071\u{b5}s iteration = 15
7212024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
7222024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7232024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7242024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7252024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7262024-05-10T21:31:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7272024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7282024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7292024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7302024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7312024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7322024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
7332024-05-10T21:31:38.789ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.832188ms iteration = 8
7342024-05-10T21:31:38.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7352024-05-10T21:31:38.790ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7362024-05-10T21:31:38.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7372024-05-10T21:31:38.790ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7382024-05-10T21:31:38.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7392024-05-10T21:31:38.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7402024-05-10T21:31:38.791ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
7412024-05-10T21:31:38.791ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.645647ms iteration = 15
7422024-05-10T21:31:38.791ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 569358457 } })
7432024-05-10T21:31:38.791ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
7442024-05-10T21:31:38.791ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.854211ms iteration = 8
7452024-05-10T21:31:38.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462024-05-10T21:31:38.792ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7472024-05-10T21:31:38.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7482024-05-10T21:31:38.793ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
7492024-05-10T21:31:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7502024-05-10T21:31:38.796ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7512024-05-10T21:31:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7522024-05-10T21:31:38.796ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7532024-05-10T21:31:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7542024-05-10T21:31:38.796ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 574187779 } })
7552024-05-10T21:31:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7562024-05-10T21:31:38.797ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
7572024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7582024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7592024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7602024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7612024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7622024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7632024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7642024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7652024-05-10T21:31:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7662024-05-10T21:31:38.798ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7672024-05-10T21:31:38.798ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7682024-05-10T21:31:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7692024-05-10T21:31:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7702024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7712024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7722024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7732024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7742024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7752024-05-10T21:31:38.799ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 577056429 } })
7762024-05-10T21:31:38.799ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
7772024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7782024-05-10T21:31:38.799ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
7792024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7802024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
7812024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
7822024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
7832024-05-10T21:31:38.799ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7842024-05-10T21:31:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7852024-05-10T21:31:38.800ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7862024-05-10T21:31:38.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7872024-05-10T21:31:38.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7882024-05-10T21:31:38.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7892024-05-10T21:31:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7902024-05-10T21:31:38.801ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
7912024-05-10T21:31:38.801ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.069444ms iteration = 15
7922024-05-10T21:31:38.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7932024-05-10T21:31:38.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942024-05-10T21:31:38.802ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7952024-05-10T21:31:38.802ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7962024-05-10T21:31:38.802ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7972024-05-10T21:31:38.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7982024-05-10T21:31:38.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7992024-05-10T21:31:38.804ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8002024-05-10T21:31:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8012024-05-10T21:31:38.804ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8022024-05-10T21:31:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8032024-05-10T21:31:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8042024-05-10T21:31:38.806ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8052024-05-10T21:31:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8062024-05-10T21:31:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
8072024-05-10T21:31:38.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8082024-05-10T21:31:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8092024-05-10T21:31:38.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8102024-05-10T21:31:38.807ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8112024-05-10T21:31:38.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8122024-05-10T21:31:38.807ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8132024-05-10T21:31:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8142024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
8152024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
8162024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8172024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8182024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8192024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8202024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8212024-05-10T21:31:38.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8222024-05-10T21:31:38.810ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8232024-05-10T21:31:38.811ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
8242024-05-10T21:31:38.811ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
8252024-05-10T21:31:38.811ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8262024-05-10T21:31:38.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8272024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8282024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
8292024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.131666ms iteration = 43
8302024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8312024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8322024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8332024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8342024-05-10T21:31:38.812ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8352024-05-10T21:31:38.813ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8362024-05-10T21:31:38.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8372024-05-10T21:31:38.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8382024-05-10T21:31:38.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8392024-05-10T21:31:38.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8402024-05-10T21:31:38.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8412024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8422024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8432024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8442024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
8452024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
8462024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
8472024-05-10T21:31:38.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.347535ms iteration = 8
8482024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
8492024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8502024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
8512024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
8522024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
8532024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 30.142141ms iteration = 8
8542024-05-10T21:31:38.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8552024-05-10T21:31:38.819ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.548491ms iteration = 15
8562024-05-10T21:31:38.829ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
8572024-05-10T21:31:38.829ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 41.054591ms iteration = 8
8582024-05-10T21:31:38.829ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
8592024-05-10T21:31:38.829ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 41.085921ms iteration = 16
8602024-05-10T21:31:38.970ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
8612024-05-10T21:31:38.970ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8622024-05-10T21:31:38.971ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d8b5c46d-c724-4494-b892-e50a2078a26d", "content-length": "26", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8632024-05-10T21:31:38.971ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8642024-05-10T21:31:38.972ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8652024-05-10T21:31:39.046ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
8662024-05-10T21:31:39.058ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
8672024-05-10T21:31:39.058ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8682024-05-10T21:31:39.058ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "52b996ed-f696-4f78-9904-167ac1740d40", "content-length": "26", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8692024-05-10T21:31:39.058ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8702024-05-10T21:31:39.060ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8712024-05-10T21:31:39.079ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
8722024-05-10T21:31:39.115ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
8732024-05-10T21:31:39.115ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8742024-05-10T21:31:39.115ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dc34e3e0-a766-4720-a995-1c4cc6c4e9b1", "content-length": "26", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8752024-05-10T21:31:39.115ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8762024-05-10T21:31:39.118ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
8772024-05-10T21:31:39.127ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 338.933065ms iteration = 15
8782024-05-10T21:31:39.214ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
8792024-05-10T21:31:39.318ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8802024-05-10T21:31:39.318ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a483c989-bc06-44fc-ae26-e493d63e6b57", "content-length": "2", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8812024-05-10T21:31:39.320ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8822024-05-10T21:31:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8832024-05-10T21:31:39.322ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 534.127207ms iteration = 15
8842024-05-10T21:31:39.329ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
8852024-05-10T21:31:39.329ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "35fe581a-9fd1-47e3-8aaa-7550d9b963aa", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8862024-05-10T21:31:39.329ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8872024-05-10T21:31:39.331ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "77ed1206-eec2-4e73-989a-be7af4ecf6f2", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8882024-05-10T21:31:39.331ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8892024-05-10T21:31:39.332ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8902024-05-10T21:31:39.332ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 544.686148ms iteration = 10
8912024-05-10T21:31:39.335ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
8922024-05-10T21:31:39.336ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "214153fd-977f-43ef-9c18-0dd60f2bacaa", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8932024-05-10T21:31:39.336ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8942024-05-10T21:31:39.338ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e30a8e67-dae1-4dd4-81fc-d79469e29ac3", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8952024-05-10T21:31:39.338ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8962024-05-10T21:31:39.339ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f413755c-4f06-42a8-8b0f-294d3bfdf643", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8972024-05-10T21:31:39.339ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8982024-05-10T21:31:39.341ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8992024-05-10T21:31:39.341ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 553.089814ms iteration = 10
9002024-05-10T21:31:39.372ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9012024-05-10T21:31:39.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9022024-05-10T21:31:39.374ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9032024-05-10T21:31:39.374ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9042024-05-10T21:31:39.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9052024-05-10T21:31:39.381ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9062024-05-10T21:31:39.381ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7f6a92c0-939b-43bd-a1b2-2cc87b2b5e91", "content-length": "121", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9072024-05-10T21:31:39.381ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
9082024-05-10T21:31:39.381ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
9092024-05-10T21:31:39.382ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9102024-05-10T21:31:39.382ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
9112024-05-10T21:31:39.382ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
9122024-05-10T21:31:39.382ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9132024-05-10T21:31:39.382ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
9142024-05-10T21:31:39.382ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
9152024-05-10T21:31:39.382ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
9162024-05-10T21:31:39.382ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9172024-05-10T21:31:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9182024-05-10T21:31:39.383ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9192024-05-10T21:31:39.383ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "04a5a52c-df18-4e25-8baf-58c812f579dc", "content-length": "2", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9202024-05-10T21:31:39.383ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9212024-05-10T21:31:39.383ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f0e6f4a4-503e-4373-8475-5be52563b902", "content-length": "51", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9222024-05-10T21:31:39.383ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9232024-05-10T21:31:39.383ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9242024-05-10T21:31:39.428ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6de3f408-f137-4488-bfa6-4bd7f1e7bfa9", "date": "Fri, 10 May 2024 21:31:39 GMT"} })
9252024-05-10T21:31:39.428ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
9262024-05-10T21:31:39.428ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9272024-05-10T21:31:39.428ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "257d47f1-a0f6-4078-bd90-1cbe694fcaa2", "content-length": "467", "date": "Fri, 10 May 2024 21:31:39 GMT"} })
9282024-05-10T21:31:39.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9292024-05-10T21:31:39.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9302024-05-10T21:31:39.432ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9312024-05-10T21:31:39.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9322024-05-10T21:31:39.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9332024-05-10T21:31:39.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9342024-05-10T21:31:39.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9352024-05-10T21:31:39.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9362024-05-10T21:31:39.442ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 654.104291ms iteration = 15
9372024-05-10T21:31:39.524ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 78 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = d0f16594-96cd-4ce5-a404-943bf8065f03 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
9382024-05-10T21:31:48.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
9392024-05-10T21:31:48.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9402024-05-10T21:31:48.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9412024-05-10T21:31:48.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9422024-05-10T21:31:48.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9432024-05-10T21:31:48.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9442024-05-10T21:31:48.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9452024-05-10T21:31:48.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9462024-05-10T21:31:48.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9472024-05-10T21:31:48.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9482024-05-10T21:31:48.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
9492024-05-10T21:31:48.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9502024-05-10T21:31:48.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9512024-05-10T21:31:48.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
9522024-05-10T21:31:48.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.675709ms iteration = 44
9532024-05-10T21:31:49.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 140 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 31e7089c-bce7-41b6-ade9-bc61307af9d2 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
9542024-05-10T21:31:58.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
9552024-05-10T21:31:58.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9562024-05-10T21:31:58.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9572024-05-10T21:31:58.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9582024-05-10T21:31:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9592024-05-10T21:31:58.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9602024-05-10T21:31:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9612024-05-10T21:31:58.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9622024-05-10T21:31:58.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9632024-05-10T21:31:58.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9642024-05-10T21:31:58.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
9652024-05-10T21:31:58.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9662024-05-10T21:31:58.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9672024-05-10T21:31:58.804ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
9682024-05-10T21:31:58.804ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.372013ms iteration = 45
9692024-05-10T21:31:59.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 130 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = cdc6425c-ea2c-48e0-935d-e551fc2d5ad4 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
9702024-05-10T21:32:08.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
9712024-05-10T21:32:08.787ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
9722024-05-10T21:32:08.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9732024-05-10T21:32:08.787ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9742024-05-10T21:32:08.787ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
9752024-05-10T21:32:08.787ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9762024-05-10T21:32:08.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
9772024-05-10T21:32:08.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
9782024-05-10T21:32:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9792024-05-10T21:32:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9802024-05-10T21:32:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9812024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
9822024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
9832024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
9842024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9852024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9862024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9872024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
9882024-05-10T21:32:08.789ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
9892024-05-10T21:32:08.789ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
9902024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.071\u{b5}s iteration = 16
9912024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
9922024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9932024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9942024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9952024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9962024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9972024-05-10T21:32:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9982024-05-10T21:32:08.790ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1819, tv_nsec: 567543276 } })
9992024-05-10T21:32:08.790ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
10002024-05-10T21:32:08.790ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.489173ms iteration = 16
10012024-05-10T21:32:08.790ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
10022024-05-10T21:32:08.790ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10032024-05-10T21:32:08.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10042024-05-10T21:32:08.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10052024-05-10T21:32:08.791ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1819, tv_nsec: 568749492 } })
10062024-05-10T21:32:08.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10072024-05-10T21:32:08.792ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
10082024-05-10T21:32:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10092024-05-10T21:32:08.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10102024-05-10T21:32:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10112024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10122024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10132024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10142024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10152024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10162024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10172024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10182024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10192024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10202024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10212024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10222024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10232024-05-10T21:32:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10242024-05-10T21:32:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10252024-05-10T21:32:08.798ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
10262024-05-10T21:32:08.798ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10272024-05-10T21:32:08.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10282024-05-10T21:32:08.798ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10292024-05-10T21:32:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10302024-05-10T21:32:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10312024-05-10T21:32:08.799ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1819, tv_nsec: 576689618 } })
10322024-05-10T21:32:08.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332024-05-10T21:32:08.800ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
10342024-05-10T21:32:08.800ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
10352024-05-10T21:32:08.800ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 11.207475ms iteration = 16
10362024-05-10T21:32:08.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
10372024-05-10T21:32:08.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10382024-05-10T21:32:08.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10392024-05-10T21:32:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10402024-05-10T21:32:08.802ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10412024-05-10T21:32:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10422024-05-10T21:32:08.805ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
10432024-05-10T21:32:08.805ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.318472ms iteration = 17
10442024-05-10T21:32:08.806ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
10452024-05-10T21:32:08.806ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.672891ms iteration = 46
10462024-05-10T21:32:08.811ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10472024-05-10T21:32:08.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10482024-05-10T21:32:08.812ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10492024-05-10T21:32:08.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10502024-05-10T21:32:08.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10512024-05-10T21:32:08.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
10522024-05-10T21:32:08.816ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10532024-05-10T21:32:08.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10542024-05-10T21:32:08.817ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.497828ms iteration = 16
10552024-05-10T21:32:08.877ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
10562024-05-10T21:32:08.877ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10572024-05-10T21:32:08.877ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "22b43fd7-5b93-449f-8ed1-bcc97688e3d4", "content-length": "26", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10582024-05-10T21:32:08.877ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10592024-05-10T21:32:08.878ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
10602024-05-10T21:32:08.878ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10612024-05-10T21:32:08.879ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a88edfc8-ac1c-46ff-b6dd-fcfb730769ab", "content-length": "26", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10622024-05-10T21:32:08.879ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10632024-05-10T21:32:08.880ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10642024-05-10T21:32:08.881ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10652024-05-10T21:32:08.894ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
10662024-05-10T21:32:08.894ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10672024-05-10T21:32:08.894ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "117db053-2980-4ef9-9752-b44221f4a2b1", "content-length": "26", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10682024-05-10T21:32:08.894ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10692024-05-10T21:32:08.896ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
10702024-05-10T21:32:08.975ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10712024-05-10T21:32:08.975ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70f24abf-a18d-4d9d-9bf3-1e54ea55e3c9", "content-length": "2", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10722024-05-10T21:32:08.977ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10732024-05-10T21:32:08.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10742024-05-10T21:32:08.978ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 190.864531ms iteration = 16
10752024-05-10T21:32:08.980ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 192.898004ms iteration = 16
10762024-05-10T21:32:09.040ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10772024-05-10T21:32:09.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10782024-05-10T21:32:09.042ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10792024-05-10T21:32:09.043ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10802024-05-10T21:32:09.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10812024-05-10T21:32:09.051ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
10822024-05-10T21:32:09.051ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "32450f68-e0f7-430f-b4d4-59a5ec1e426a", "content-length": "121", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10832024-05-10T21:32:09.051ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
10842024-05-10T21:32:09.051ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
10852024-05-10T21:32:09.052ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10862024-05-10T21:32:09.052ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
10872024-05-10T21:32:09.052ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
10882024-05-10T21:32:09.052ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10892024-05-10T21:32:09.052ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
10902024-05-10T21:32:09.052ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
10912024-05-10T21:32:09.052ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
10922024-05-10T21:32:09.052ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10932024-05-10T21:32:09.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10942024-05-10T21:32:09.054ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
10952024-05-10T21:32:09.054ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5839bace-ec3f-4192-b6e0-f2ed30f6e99c", "content-length": "2", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10962024-05-10T21:32:09.054ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
10972024-05-10T21:32:09.054ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "69da1363-e089-48ce-b24e-5dbc97ad60fd", "content-length": "51", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10982024-05-10T21:32:09.054ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10992024-05-10T21:32:09.054ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
11002024-05-10T21:32:09.097ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3f18af24-74cc-473a-8f4a-ac6fb9b4a05c", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
11012024-05-10T21:32:09.097ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
11022024-05-10T21:32:09.097ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
11032024-05-10T21:32:09.097ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3814c416-17fc-4a30-991c-6b6b9ae63543", "content-length": "467", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
11042024-05-10T21:32:09.097ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11052024-05-10T21:32:09.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11062024-05-10T21:32:09.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11072024-05-10T21:32:09.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11082024-05-10T21:32:09.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11092024-05-10T21:32:09.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11102024-05-10T21:32:09.104ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11112024-05-10T21:32:09.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11122024-05-10T21:32:09.109ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 320.447687ms iteration = 16
11132024-05-10T21:32:09.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 138 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 95f75d26-8ad5-47a8-a5c9-094693ff7b88 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
11142024-05-10T21:32:18.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
11152024-05-10T21:32:18.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11162024-05-10T21:32:18.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11172024-05-10T21:32:18.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11182024-05-10T21:32:18.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11192024-05-10T21:32:18.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11202024-05-10T21:32:18.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11212024-05-10T21:32:18.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11222024-05-10T21:32:18.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11232024-05-10T21:32:18.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11242024-05-10T21:32:18.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
11252024-05-10T21:32:18.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11262024-05-10T21:32:18.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11272024-05-10T21:32:18.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
11282024-05-10T21:32:18.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.495875ms iteration = 47
11292024-05-10T21:32:19.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 129 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 8523a3eb-d6ae-4838-a498-026396a55625 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
11302024-05-10T21:32:28.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
11312024-05-10T21:32:28.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11322024-05-10T21:32:28.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11332024-05-10T21:32:28.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11342024-05-10T21:32:28.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11352024-05-10T21:32:28.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11362024-05-10T21:32:28.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11372024-05-10T21:32:28.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11382024-05-10T21:32:28.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11392024-05-10T21:32:28.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11402024-05-10T21:32:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
11412024-05-10T21:32:28.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11422024-05-10T21:32:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11432024-05-10T21:32:28.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
11442024-05-10T21:32:28.804ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.681499ms iteration = 48
11452024-05-10T21:32:29.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 127 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = a186c004-9c38-409f-a5c5-f00443fd065c response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
11462024-05-10T21:32:38.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
11472024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
11482024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11492024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
11502024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11512024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
11522024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
11532024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
11542024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
11552024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11562024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
11572024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
11582024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
11592024-05-10T21:32:38.788ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
11602024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11612024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.32\u{b5}s iteration = 9
11622024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
11632024-05-10T21:32:38.788ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
11642024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.59\u{b5}s iteration = 10
11652024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
11662024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
11672024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
11682024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
11692024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
11702024-05-10T21:32:38.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
11712024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11722024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11732024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11742024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11752024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11762024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
11772024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11782024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
11792024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11802024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
11812024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
11822024-05-10T21:32:38.788ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
11832024-05-10T21:32:38.788ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
11842024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.18\u{b5}s iteration = 17
11852024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
11862024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11872024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11882024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
11892024-05-10T21:32:38.788ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11902024-05-10T21:32:38.788ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 566282497 } })
11912024-05-10T21:32:38.788ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 566282437 } })
11922024-05-10T21:32:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11932024-05-10T21:32:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11942024-05-10T21:32:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11952024-05-10T21:32:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11962024-05-10T21:32:38.789ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11982024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11992024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12012024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12022024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
12032024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.87482ms iteration = 9
12042024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
12052024-05-10T21:32:38.790ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.959782ms iteration = 9
12062024-05-10T21:32:38.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12072024-05-10T21:32:38.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12082024-05-10T21:32:38.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12092024-05-10T21:32:38.792ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
12102024-05-10T21:32:38.792ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.930152ms iteration = 17
12112024-05-10T21:32:38.792ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 569591296 } })
12122024-05-10T21:32:38.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12132024-05-10T21:32:38.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12142024-05-10T21:32:38.793ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
12152024-05-10T21:32:38.794ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12162024-05-10T21:32:38.794ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12172024-05-10T21:32:38.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12182024-05-10T21:32:38.794ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 572075459 } })
12192024-05-10T21:32:38.794ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
12202024-05-10T21:32:38.795ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12212024-05-10T21:32:38.796ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12222024-05-10T21:32:38.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12232024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12242024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12252024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12262024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12272024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12282024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12292024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12302024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12312024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12322024-05-10T21:32:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
12332024-05-10T21:32:38.798ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12342024-05-10T21:32:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12352024-05-10T21:32:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12362024-05-10T21:32:38.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12372024-05-10T21:32:38.799ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 576990411 } })
12382024-05-10T21:32:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12392024-05-10T21:32:38.799ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12402024-05-10T21:32:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12412024-05-10T21:32:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12422024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12432024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12442024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12452024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12462024-05-10T21:32:38.800ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
12472024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12482024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12492024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12502024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12512024-05-10T21:32:38.800ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12522024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12532024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12542024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12552024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12562024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12572024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12582024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12592024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12602024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12612024-05-10T21:32:38.801ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
12622024-05-10T21:32:38.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12632024-05-10T21:32:38.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12642024-05-10T21:32:38.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12652024-05-10T21:32:38.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12662024-05-10T21:32:38.802ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
12672024-05-10T21:32:38.803ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.2901ms iteration = 17
12682024-05-10T21:32:38.803ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12692024-05-10T21:32:38.803ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12702024-05-10T21:32:38.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12712024-05-10T21:32:38.804ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12722024-05-10T21:32:38.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
12732024-05-10T21:32:38.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12742024-05-10T21:32:38.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12752024-05-10T21:32:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12762024-05-10T21:32:38.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12772024-05-10T21:32:38.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12782024-05-10T21:32:38.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12792024-05-10T21:32:38.807ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12802024-05-10T21:32:38.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12812024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12822024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12832024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12842024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12852024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12862024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12872024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12882024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12892024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12902024-05-10T21:32:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12912024-05-10T21:32:38.809ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12922024-05-10T21:32:38.809ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12932024-05-10T21:32:38.809ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
12942024-05-10T21:32:38.809ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12952024-05-10T21:32:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12962024-05-10T21:32:38.810ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
12972024-05-10T21:32:38.810ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.492312ms iteration = 49
12982024-05-10T21:32:38.810ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
12992024-05-10T21:32:38.810ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.994551ms iteration = 18
13002024-05-10T21:32:38.811ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13012024-05-10T21:32:38.812ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13022024-05-10T21:32:38.812ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13032024-05-10T21:32:38.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13042024-05-10T21:32:38.812ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13052024-05-10T21:32:38.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13062024-05-10T21:32:38.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13072024-05-10T21:32:38.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13082024-05-10T21:32:38.814ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
13092024-05-10T21:32:38.814ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
13102024-05-10T21:32:38.814ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
13112024-05-10T21:32:38.814ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.793122ms iteration = 9
13122024-05-10T21:32:38.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13132024-05-10T21:32:38.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13142024-05-10T21:32:38.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13152024-05-10T21:32:38.815ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13162024-05-10T21:32:38.816ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
13172024-05-10T21:32:38.816ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13182024-05-10T21:32:38.816ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
13192024-05-10T21:32:38.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.60234ms iteration = 9
13202024-05-10T21:32:38.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
13212024-05-10T21:32:38.818ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13222024-05-10T21:32:38.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13232024-05-10T21:32:38.820ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.51484ms iteration = 17
13242024-05-10T21:32:38.844ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
13252024-05-10T21:32:38.844ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 56.403792ms iteration = 9
13262024-05-10T21:32:38.923ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
13272024-05-10T21:32:39.055ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13282024-05-10T21:32:39.059ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13292024-05-10T21:32:39.061ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "64c6df72-6267-4f6a-9a59-1b43b2680894", "date": "Fri, 10 May 2024 21:32:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13302024-05-10T21:32:39.061ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13312024-05-10T21:32:39.062ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b6e017de-5169-48a0-bfc7-3d76c9c45356", "date": "Fri, 10 May 2024 21:32:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13322024-05-10T21:32:39.062ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13332024-05-10T21:32:39.065ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13342024-05-10T21:32:39.065ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 277.741552ms iteration = 11
13352024-05-10T21:32:39.071ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
13362024-05-10T21:32:39.071ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13372024-05-10T21:32:39.072ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7b1fcae0-623b-4059-a536-ca8cb78dee86", "content-length": "26", "date": "Fri, 10 May 2024 21:32:38 GMT"} })
13382024-05-10T21:32:39.072ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13392024-05-10T21:32:39.074ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13402024-05-10T21:32:39.079ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
13412024-05-10T21:32:39.079ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13422024-05-10T21:32:39.079ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1e93e952-e35f-4b48-a0be-f1f264e5bdc6", "content-length": "26", "date": "Fri, 10 May 2024 21:32:38 GMT"} })
13432024-05-10T21:32:39.080ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13442024-05-10T21:32:39.082ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13452024-05-10T21:32:39.082ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
13462024-05-10T21:32:39.082ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13472024-05-10T21:32:39.083ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "90ffe334-b572-4bb7-b6cc-5079a16ed56d", "content-length": "26", "date": "Fri, 10 May 2024 21:32:38 GMT"} })
13482024-05-10T21:32:39.083ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13492024-05-10T21:32:39.084ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
13502024-05-10T21:32:39.205ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13512024-05-10T21:32:39.206ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "67f9089b-fd0f-4670-8ab4-10a3060a67d1", "content-length": "2", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13522024-05-10T21:32:39.207ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13532024-05-10T21:32:39.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13542024-05-10T21:32:39.209ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 420.940354ms iteration = 17
13552024-05-10T21:32:39.215ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 427.720017ms iteration = 17
13562024-05-10T21:32:39.252ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13572024-05-10T21:32:39.369ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13582024-05-10T21:32:39.370ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "10975491-83b4-47f7-897a-0076cbd4230e", "date": "Fri, 10 May 2024 21:32:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13592024-05-10T21:32:39.370ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13602024-05-10T21:32:39.371ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "78151547-8d10-4c4e-ad83-9b85d55c3f29", "date": "Fri, 10 May 2024 21:32:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13612024-05-10T21:32:39.371ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13622024-05-10T21:32:39.372ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ea362b31-43ee-41a2-8c48-7b29be5510ef", "date": "Fri, 10 May 2024 21:32:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13632024-05-10T21:32:39.372ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13642024-05-10T21:32:39.374ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13652024-05-10T21:32:39.374ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 586.322051ms iteration = 11
13662024-05-10T21:32:39.404ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13672024-05-10T21:32:39.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13682024-05-10T21:32:39.406ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13692024-05-10T21:32:39.406ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13702024-05-10T21:32:39.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13712024-05-10T21:32:39.412ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13722024-05-10T21:32:39.413ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b02f13c7-2b61-4fba-90f5-af6950f3d83f", "content-length": "121", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13732024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13742024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
13752024-05-10T21:32:39.413ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13762024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
13772024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
13782024-05-10T21:32:39.413ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13792024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
13802024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
13812024-05-10T21:32:39.413ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
13822024-05-10T21:32:39.413ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13832024-05-10T21:32:39.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13842024-05-10T21:32:39.415ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13852024-05-10T21:32:39.415ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "04b7f32e-d6cf-4c92-9880-fc95ba7347f9", "content-length": "2", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13862024-05-10T21:32:39.415ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13872024-05-10T21:32:39.416ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b63fe42c-538c-4161-83e8-ca832b37b342", "content-length": "51", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13882024-05-10T21:32:39.416ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13892024-05-10T21:32:39.416ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13902024-05-10T21:32:39.458ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5aabafcc-9ffe-4e02-90eb-83f8f72190f3", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13912024-05-10T21:32:39.458ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
13922024-05-10T21:32:39.458ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13932024-05-10T21:32:39.458ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6f026cff-1a41-4912-a4b3-365cf08fefb1", "content-length": "467", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13942024-05-10T21:32:39.458ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13952024-05-10T21:32:39.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13962024-05-10T21:32:39.462ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13972024-05-10T21:32:39.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13982024-05-10T21:32:39.464ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13992024-05-10T21:32:39.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14002024-05-10T21:32:39.465ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14012024-05-10T21:32:39.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14022024-05-10T21:32:39.470ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 681.900545ms iteration = 17
14032024-05-10T21:32:39.524ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 73 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 444b1f80-0fb2-4538-8cf6-7534d7cddeea response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
14042024-05-10T21:32:48.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
14052024-05-10T21:32:48.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14062024-05-10T21:32:48.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14072024-05-10T21:32:48.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14082024-05-10T21:32:48.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14092024-05-10T21:32:48.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14102024-05-10T21:32:48.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14112024-05-10T21:32:48.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14122024-05-10T21:32:48.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14132024-05-10T21:32:48.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14142024-05-10T21:32:48.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
14152024-05-10T21:32:48.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14162024-05-10T21:32:48.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14172024-05-10T21:32:48.804ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
14182024-05-10T21:32:48.804ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.435473ms iteration = 50
14192024-05-10T21:32:49.524ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 97 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 1509db36-ebaa-4edf-970f-be9dc52c58c2 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
14202024-05-10T21:32:58.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
14212024-05-10T21:32:58.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14222024-05-10T21:32:58.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14232024-05-10T21:32:58.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14242024-05-10T21:32:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14252024-05-10T21:32:58.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14262024-05-10T21:32:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14272024-05-10T21:32:58.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14282024-05-10T21:32:58.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14292024-05-10T21:32:58.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14302024-05-10T21:32:58.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
14312024-05-10T21:32:58.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14322024-05-10T21:32:58.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14332024-05-10T21:32:58.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
14342024-05-10T21:32:58.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.874823ms iteration = 51
14352024-05-10T21:32:59.524ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 129 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 4a3e49d3-f3e4-4880-8105-b082c9b72f03 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
14362024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
14372024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14382024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
14392024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
14402024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14412024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14422024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
14432024-05-10T21:33:08.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
14442024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14452024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14462024-05-10T21:33:08.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14472024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
14482024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
14492024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14502024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
14512024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14522024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14532024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
14542024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
14552024-05-10T21:33:08.789ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
14562024-05-10T21:33:08.789ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
14572024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.83\u{b5}s iteration = 18
14582024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14592024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14602024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14612024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14622024-05-10T21:33:08.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14632024-05-10T21:33:08.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14642024-05-10T21:33:08.790ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1879, tv_nsec: 568401909 } })
14652024-05-10T21:33:08.791ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14662024-05-10T21:33:08.791ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672024-05-10T21:33:08.791ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1879, tv_nsec: 568566512 } })
14682024-05-10T21:33:08.791ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
14692024-05-10T21:33:08.791ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.974432ms iteration = 18
14702024-05-10T21:33:08.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712024-05-10T21:33:08.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14722024-05-10T21:33:08.792ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
14732024-05-10T21:33:08.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14742024-05-10T21:33:08.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14752024-05-10T21:33:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14762024-05-10T21:33:08.795ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
14772024-05-10T21:33:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14782024-05-10T21:33:08.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14792024-05-10T21:33:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
14802024-05-10T21:33:08.795ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14812024-05-10T21:33:08.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14822024-05-10T21:33:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14832024-05-10T21:33:08.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14842024-05-10T21:33:08.796ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14852024-05-10T21:33:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14862024-05-10T21:33:08.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14872024-05-10T21:33:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14882024-05-10T21:33:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14892024-05-10T21:33:08.797ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14902024-05-10T21:33:08.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14912024-05-10T21:33:08.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14922024-05-10T21:33:08.798ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
14932024-05-10T21:33:08.798ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14942024-05-10T21:33:08.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14952024-05-10T21:33:08.798ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1879, tv_nsec: 576365916 } })
14962024-05-10T21:33:08.799ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14972024-05-10T21:33:08.799ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
14982024-05-10T21:33:08.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14992024-05-10T21:33:08.799ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
15002024-05-10T21:33:08.799ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.5344ms iteration = 18
15012024-05-10T21:33:08.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15022024-05-10T21:33:08.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
15032024-05-10T21:33:08.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15042024-05-10T21:33:08.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15052024-05-10T21:33:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15062024-05-10T21:33:08.802ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15072024-05-10T21:33:08.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15082024-05-10T21:33:08.805ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
15092024-05-10T21:33:08.805ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.058367ms iteration = 19
15102024-05-10T21:33:08.806ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
15112024-05-10T21:33:08.806ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.175901ms iteration = 52
15122024-05-10T21:33:08.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15132024-05-10T21:33:08.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15142024-05-10T21:33:08.816ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15152024-05-10T21:33:08.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15162024-05-10T21:33:08.817ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15172024-05-10T21:33:08.819ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
15182024-05-10T21:33:08.819ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15192024-05-10T21:33:08.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15202024-05-10T21:33:08.821ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.803847ms iteration = 18
15212024-05-10T21:33:08.895ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
15222024-05-10T21:33:08.895ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15232024-05-10T21:33:08.895ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d6897f61-885f-4f30-a901-6f2bf1977608", "content-length": "26", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15242024-05-10T21:33:08.895ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15252024-05-10T21:33:08.897ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15262024-05-10T21:33:08.999ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15272024-05-10T21:33:08.999ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e0210943-6723-4bc3-88dc-1f3b08dfbc91", "content-length": "2", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15282024-05-10T21:33:09.001ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
15292024-05-10T21:33:09.001ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15302024-05-10T21:33:09.001ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ea2cb7c4-d241-44ee-a733-c77e6e208475", "content-length": "26", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15312024-05-10T21:33:09.001ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15322024-05-10T21:33:09.001ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
15332024-05-10T21:33:09.001ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15342024-05-10T21:33:09.001ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15352024-05-10T21:33:09.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15362024-05-10T21:33:09.002ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "79525c59-bda7-47e1-8eb5-f2ae38152b51", "content-length": "26", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15372024-05-10T21:33:09.002ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15382024-05-10T21:33:09.002ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15392024-05-10T21:33:09.003ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 214.942646ms iteration = 18
15402024-05-10T21:33:09.004ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
15412024-05-10T21:33:09.066ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 278.59663ms iteration = 18
15422024-05-10T21:33:09.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15432024-05-10T21:33:09.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15442024-05-10T21:33:09.125ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15452024-05-10T21:33:09.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15462024-05-10T21:33:09.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15472024-05-10T21:33:09.131ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15482024-05-10T21:33:09.132ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "74e59f18-c915-4b3a-bb3e-88148cd4039d", "content-length": "121", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
15492024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
15502024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
15512024-05-10T21:33:09.132ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15522024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
15532024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
15542024-05-10T21:33:09.132ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15552024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
15562024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
15572024-05-10T21:33:09.132ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
15582024-05-10T21:33:09.132ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15592024-05-10T21:33:09.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15602024-05-10T21:33:09.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15612024-05-10T21:33:09.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "96b77870-90c0-47be-b3d5-f0e99710de98", "content-length": "2", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
15622024-05-10T21:33:09.134ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15632024-05-10T21:33:09.134ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a6a9b1ef-61ba-47f0-9ca1-5b6955118ff8", "content-length": "51", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
15642024-05-10T21:33:09.134ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15652024-05-10T21:33:09.134ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15662024-05-10T21:33:09.212ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1fdbd863-e620-4d56-a100-98ca158e97b0", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15672024-05-10T21:33:09.212ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
15682024-05-10T21:33:09.212ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15692024-05-10T21:33:09.212ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dd391f98-7b3f-49c0-b341-9be499988d53", "content-length": "467", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
15702024-05-10T21:33:09.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15712024-05-10T21:33:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15722024-05-10T21:33:09.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15732024-05-10T21:33:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15742024-05-10T21:33:09.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15752024-05-10T21:33:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15762024-05-10T21:33:09.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15772024-05-10T21:33:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15782024-05-10T21:33:09.225ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 435.901038ms iteration = 18
15792024-05-10T21:33:09.525ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 91 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 65ef7cfd-9faf-4f06-b3de-f88edd5ab27a response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
15802024-05-10T21:33:09.529ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
15812024-05-10T21:33:09.529ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
15822024-05-10T21:33:09.558ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
15832024-05-10T21:33:09.559ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:49178
15842024-05-10T21:33:09.559ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:49178 req_id = 1092edb3-3eab-4424-b4c2-248ae1e30d2a roles = RoleSet { roles: {} } uri = /metrics/producers
15852024-05-10T21:33:09.559ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:49178 req_id = 1092edb3-3eab-4424-b4c2-248ae1e30d2a resource = Database result = Ok(()) uri = /metrics/producers
15862024-05-10T21:33:09.590ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 95738557-5fbe-4760-a2f5-e295833eb800
15872024-05-10T21:33:09.590ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:49178 req_id = 1092edb3-3eab-4424-b4c2-248ae1e30d2a roles = RoleSet { roles: {} } uri = /metrics/producers
15882024-05-10T21:33:09.591ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:49178 req_id = 1092edb3-3eab-4424-b4c2-248ae1e30d2a resource = Database result = Ok(()) uri = /metrics/producers
15892024-05-10T21:33:09.622ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 uri = http://[fd00:1122:3344:101::d]:12223/producers
15902024-05-10T21:33:09.623ZDEBGnexus: client response oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0a53e99d-436f-4901-b0ed-b7049c5b99d9", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
15912024-05-10T21:33:09.623ZINFOnexus: assigned collector to new producer collector_id = 95738557-5fbe-4760-a2f5-e295833eb800 file = nexus/src/app/oximeter.rs:125 producer_id = 7bb2217a-5bab-47ca-8e9e-29f166894592
15922024-05-10T21:33:09.624ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 65795 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:49178 req_id = 1092edb3-3eab-4424-b4c2-248ae1e30d2a response_code = 201 uri = /metrics/producers
15932024-05-10T21:33:09.624ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "1092edb3-3eab-4424-b4c2-248ae1e30d2a", "content-length": "41", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
15942024-05-10T21:33:09.626ZDEBGnexus (producer-registration-task): registered with nexus successfully
15952024-05-10T21:33:09.626ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
15962024-05-10T21:33:18.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
15972024-05-10T21:33:18.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15982024-05-10T21:33:18.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15992024-05-10T21:33:18.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16002024-05-10T21:33:18.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16012024-05-10T21:33:18.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16022024-05-10T21:33:18.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16032024-05-10T21:33:18.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16042024-05-10T21:33:18.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16052024-05-10T21:33:18.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16062024-05-10T21:33:18.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
16072024-05-10T21:33:18.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16082024-05-10T21:33:18.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16092024-05-10T21:33:18.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
16102024-05-10T21:33:18.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.532396ms iteration = 53
16112024-05-10T21:33:19.625ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 103 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 7fba1d7d-0601-4ea5-88d5-56921edbc5a2 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
16122024-05-10T21:33:28.787ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
16132024-05-10T21:33:28.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16142024-05-10T21:33:28.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16152024-05-10T21:33:28.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16162024-05-10T21:33:28.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16172024-05-10T21:33:28.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16182024-05-10T21:33:28.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16192024-05-10T21:33:28.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16202024-05-10T21:33:28.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16212024-05-10T21:33:28.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16222024-05-10T21:33:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
16232024-05-10T21:33:28.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16242024-05-10T21:33:28.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16252024-05-10T21:33:28.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
16262024-05-10T21:33:28.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.212369ms iteration = 54
16272024-05-10T21:33:29.625ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 142 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = cad4351b-5d8f-4ae5-abcd-60a4a8e10309 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
16282024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
16292024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
16302024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16312024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16322024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16332024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
16342024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
16352024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
16362024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16372024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16382024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
16392024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16402024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
16412024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
16422024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16432024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
16442024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
16452024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
16462024-05-10T21:33:38.788ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
16472024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 11.28\u{b5}s iteration = 10
16482024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
16492024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
16502024-05-10T21:33:38.788ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
16512024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.49\u{b5}s iteration = 11
16522024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16532024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
16542024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16552024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
16562024-05-10T21:33:38.788ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
16572024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16582024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
16592024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
16602024-05-10T21:33:38.788ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
16612024-05-10T21:33:38.788ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
16622024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16632024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.78\u{b5}s iteration = 19
16642024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
16652024-05-10T21:33:38.788ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16662024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16672024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16682024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16692024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16702024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16712024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16722024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16732024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16742024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16752024-05-10T21:33:38.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16762024-05-10T21:33:38.790ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 567770153 } })
16772024-05-10T21:33:38.790ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 567984488 } })
16782024-05-10T21:33:38.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16792024-05-10T21:33:38.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16802024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16812024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16822024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16832024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16842024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16852024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16862024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16872024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
16882024-05-10T21:33:38.792ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.169417ms iteration = 10
16892024-05-10T21:33:38.793ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
16902024-05-10T21:33:38.793ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.33787ms iteration = 19
16912024-05-10T21:33:38.793ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16922024-05-10T21:33:38.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16932024-05-10T21:33:38.794ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16942024-05-10T21:33:38.794ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
16952024-05-10T21:33:38.794ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.611388ms iteration = 10
16962024-05-10T21:33:38.795ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 573326569 } })
16972024-05-10T21:33:38.796ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 573497803 } })
16982024-05-10T21:33:38.796ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
16992024-05-10T21:33:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17002024-05-10T21:33:38.797ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17012024-05-10T21:33:38.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17022024-05-10T21:33:38.797ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
17032024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17042024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17052024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17062024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17072024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17082024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17092024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17102024-05-10T21:33:38.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17112024-05-10T21:33:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17122024-05-10T21:33:38.799ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
17132024-05-10T21:33:38.799ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17142024-05-10T21:33:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17152024-05-10T21:33:38.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17162024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17172024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17182024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17192024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17202024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17212024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17222024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17232024-05-10T21:33:38.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17242024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17252024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
17262024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
17272024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
17282024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17292024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17302024-05-10T21:33:38.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17312024-05-10T21:33:38.801ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 579021569 } })
17322024-05-10T21:33:38.801ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
17332024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17342024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17352024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
17362024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
17372024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
17382024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17392024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17402024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17412024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17422024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17432024-05-10T21:33:38.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17442024-05-10T21:33:38.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17452024-05-10T21:33:38.803ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
17462024-05-10T21:33:38.803ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.945994ms iteration = 19
17472024-05-10T21:33:38.803ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17482024-05-10T21:33:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17492024-05-10T21:33:38.804ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17502024-05-10T21:33:38.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17512024-05-10T21:33:38.807ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
17522024-05-10T21:33:38.807ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 18.349645ms iteration = 20
17532024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
17542024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17552024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17562024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17572024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17582024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17592024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17602024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17612024-05-10T21:33:38.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17622024-05-10T21:33:38.809ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
17632024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
17642024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17652024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17662024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
17672024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17682024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
17692024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
17702024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17712024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17722024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17732024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17742024-05-10T21:33:38.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17752024-05-10T21:33:38.811ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17762024-05-10T21:33:38.811ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17772024-05-10T21:33:38.811ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17782024-05-10T21:33:38.815ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17792024-05-10T21:33:38.815ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17802024-05-10T21:33:38.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17812024-05-10T21:33:38.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17822024-05-10T21:33:38.815ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17832024-05-10T21:33:38.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17842024-05-10T21:33:38.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17852024-05-10T21:33:38.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17862024-05-10T21:33:38.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17872024-05-10T21:33:38.817ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
17882024-05-10T21:33:38.817ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
17892024-05-10T21:33:38.817ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
17902024-05-10T21:33:38.817ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.885485ms iteration = 10
17912024-05-10T21:33:38.818ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
17922024-05-10T21:33:38.818ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
17932024-05-10T21:33:38.818ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
17942024-05-10T21:33:38.818ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 29.690303ms iteration = 10
17952024-05-10T21:33:38.822ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17962024-05-10T21:33:38.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17972024-05-10T21:33:38.823ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17982024-05-10T21:33:38.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17992024-05-10T21:33:38.825ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002024-05-10T21:33:38.826ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
18012024-05-10T21:33:38.826ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.393345ms iteration = 55
18022024-05-10T21:33:38.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
18032024-05-10T21:33:38.828ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18042024-05-10T21:33:38.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18052024-05-10T21:33:38.829ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 41.067741ms iteration = 19
18062024-05-10T21:33:38.849ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
18072024-05-10T21:33:38.849ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 60.651011ms iteration = 10
18082024-05-10T21:33:38.941ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
18092024-05-10T21:33:38.941ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18102024-05-10T21:33:38.941ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "51929a2a-977c-4410-90df-4cdc766d327d", "content-length": "26", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18112024-05-10T21:33:38.941ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18122024-05-10T21:33:38.944ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18132024-05-10T21:33:38.953ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
18142024-05-10T21:33:38.953ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18152024-05-10T21:33:38.953ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9fe6f429-9f40-4a2e-8c2e-92dc73564715", "content-length": "26", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18162024-05-10T21:33:38.953ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18172024-05-10T21:33:38.955ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18182024-05-10T21:33:38.964ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
18192024-05-10T21:33:38.964ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18202024-05-10T21:33:38.965ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "14530247-6461-4abd-8422-ea084d2406c9", "content-length": "26", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18212024-05-10T21:33:38.965ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18222024-05-10T21:33:38.966ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
18232024-05-10T21:33:39.064ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
18242024-05-10T21:33:39.068ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
18252024-05-10T21:33:39.078ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 289.810206ms iteration = 19
18262024-05-10T21:33:39.095ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18272024-05-10T21:33:39.097ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b8862262-f0b5-4403-ae67-175305018632", "content-length": "2", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18282024-05-10T21:33:39.098ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18292024-05-10T21:33:39.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18302024-05-10T21:33:39.100ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 311.720575ms iteration = 19
18312024-05-10T21:33:39.241ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
18322024-05-10T21:33:39.266ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18332024-05-10T21:33:39.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18342024-05-10T21:33:39.267ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
18352024-05-10T21:33:39.268ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "82500d31-974b-4cfd-8c8e-a74c6d0a21d3", "date": "Fri, 10 May 2024 21:33:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18362024-05-10T21:33:39.268ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18372024-05-10T21:33:39.269ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18382024-05-10T21:33:39.269ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18392024-05-10T21:33:39.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18402024-05-10T21:33:39.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c19545e9-20dc-44c9-ab03-57c31fbfdea7", "date": "Fri, 10 May 2024 21:33:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18412024-05-10T21:33:39.269ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18422024-05-10T21:33:39.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18432024-05-10T21:33:39.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 483.25753ms iteration = 12
18442024-05-10T21:33:39.276ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
18452024-05-10T21:33:39.276ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "feec4837-eb3f-46d2-8ec3-1b68f9b78bb1", "content-length": "121", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18462024-05-10T21:33:39.276ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
18472024-05-10T21:33:39.276ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
18482024-05-10T21:33:39.277ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18492024-05-10T21:33:39.277ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
18502024-05-10T21:33:39.277ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
18512024-05-10T21:33:39.277ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18522024-05-10T21:33:39.277ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
18532024-05-10T21:33:39.277ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
18542024-05-10T21:33:39.277ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
18552024-05-10T21:33:39.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18562024-05-10T21:33:39.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18572024-05-10T21:33:39.279ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
18582024-05-10T21:33:39.279ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a05b8a-438d-4b06-8cf6-1e1d6d4274b1", "content-length": "2", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18592024-05-10T21:33:39.279ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
18602024-05-10T21:33:39.279ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "92b089bd-8a38-49f5-833b-f97bcb29639d", "content-length": "51", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18612024-05-10T21:33:39.279ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18622024-05-10T21:33:39.279ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
18632024-05-10T21:33:39.293ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
18642024-05-10T21:33:39.293ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dfc5fafe-5fa3-4a55-9ba8-15f8eee4e158", "date": "Fri, 10 May 2024 21:33:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18652024-05-10T21:33:39.293ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18662024-05-10T21:33:39.294ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0f56198e-f617-40a4-bd3e-37c71151d099", "date": "Fri, 10 May 2024 21:33:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18672024-05-10T21:33:39.294ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18682024-05-10T21:33:39.296ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2ce81a50-6eb6-41f7-98c4-99f02100b79d", "date": "Fri, 10 May 2024 21:33:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18692024-05-10T21:33:39.296ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18702024-05-10T21:33:39.297ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18712024-05-10T21:33:39.297ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 509.181044ms iteration = 12
18722024-05-10T21:33:39.333ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "566da2e4-f236-45fc-9ba3-101b80d7b014", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18732024-05-10T21:33:39.333ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
18742024-05-10T21:33:39.333ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
18752024-05-10T21:33:39.336ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2ad41dad-a186-48da-9d8c-33ea4cde0dc3", "content-length": "467", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18762024-05-10T21:33:39.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18772024-05-10T21:33:39.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18782024-05-10T21:33:39.340ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18792024-05-10T21:33:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18802024-05-10T21:33:39.342ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18812024-05-10T21:33:39.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18822024-05-10T21:33:39.344ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18832024-05-10T21:33:39.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18842024-05-10T21:33:39.352ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 563.38672ms iteration = 19
18852024-05-10T21:33:39.625ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 133 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 8ada2836-c2d8-435d-9b8e-362113471c9b response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
18862024-05-10T21:33:48.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
18872024-05-10T21:33:48.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18882024-05-10T21:33:48.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18892024-05-10T21:33:48.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18902024-05-10T21:33:48.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18912024-05-10T21:33:48.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18922024-05-10T21:33:48.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18932024-05-10T21:33:48.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18942024-05-10T21:33:48.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18952024-05-10T21:33:48.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18962024-05-10T21:33:48.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
18972024-05-10T21:33:48.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18982024-05-10T21:33:48.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18992024-05-10T21:33:48.804ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
19002024-05-10T21:33:48.804ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.585937ms iteration = 56
19012024-05-10T21:33:49.624ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 96 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 2ed0c6e8-1545-42b1-8736-08db71327fe8 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592
19022024-05-10T21:33:58.788ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
19032024-05-10T21:33:58.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19042024-05-10T21:33:58.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19052024-05-10T21:33:58.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062024-05-10T21:33:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19072024-05-10T21:33:58.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19082024-05-10T21:33:58.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19092024-05-10T21:33:58.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19102024-05-10T21:33:58.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19112024-05-10T21:33:58.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122024-05-10T21:33:58.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
19132024-05-10T21:33:58.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19142024-05-10T21:33:58.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19152024-05-10T21:33:58.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
19162024-05-10T21:33:58.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.092286ms iteration = 57
19172024-05-10T21:33:59.624ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 85 local_addr = [fd00:1122:3344:101::b]:39219 method = GET remote_addr = [fd00:1122:3344:101::d]:48935 req_id = 18da7e0e-b211-42b4-a22c-11ab3ab03b96 response_code = 200 uri = //7bb2217a-5bab-47ca-8e9e-29f166894592