Max level shown:
12024-05-10T21:30:08.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
22024-05-10T21:30:08.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
32024-05-10T21:30:08.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42024-05-10T21:30:08.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
52024-05-10T21:30:08.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62024-05-10T21:30:08.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
72024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
82024-05-10T21:30:08.861ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
92024-05-10T21:30:08.861ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
102024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
112024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.75\u{b5}s iteration = 13
122024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
132024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
142024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
152024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
172024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
182024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
202024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
212024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
222024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
242024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
252024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
262024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
272024-05-10T21:30:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
282024-05-10T21:30:08.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
292024-05-10T21:30:08.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
302024-05-10T21:30:08.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
312024-05-10T21:30:08.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.491723ms iteration = 13
322024-05-10T21:30:08.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
332024-05-10T21:30:08.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
342024-05-10T21:30:08.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
352024-05-10T21:30:08.864ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1699, tv_nsec: 641999559 } })
362024-05-10T21:30:08.864ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1699, tv_nsec: 642210223 } })
372024-05-10T21:30:08.865ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
382024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
392024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
402024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
412024-05-10T21:30:08.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
422024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
432024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
442024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
452024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
462024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
472024-05-10T21:30:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
482024-05-10T21:30:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
492024-05-10T21:30:08.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
502024-05-10T21:30:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
512024-05-10T21:30:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
522024-05-10T21:30:08.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
532024-05-10T21:30:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
542024-05-10T21:30:08.869ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
552024-05-10T21:30:08.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
562024-05-10T21:30:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
572024-05-10T21:30:08.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
582024-05-10T21:30:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
592024-05-10T21:30:08.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
602024-05-10T21:30:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
612024-05-10T21:30:08.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
622024-05-10T21:30:08.870ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1699, tv_nsec: 648013555 } })
632024-05-10T21:30:08.870ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
642024-05-10T21:30:08.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
652024-05-10T21:30:08.872ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
662024-05-10T21:30:08.872ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 11.403499ms iteration = 13
672024-05-10T21:30:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
682024-05-10T21:30:08.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
692024-05-10T21:30:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
702024-05-10T21:30:08.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
712024-05-10T21:30:08.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
722024-05-10T21:30:08.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
732024-05-10T21:30:08.875ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
742024-05-10T21:30:08.875ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 14.947503ms iteration = 15
752024-05-10T21:30:08.879ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
762024-05-10T21:30:08.879ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.899707ms iteration = 35
772024-05-10T21:30:08.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
782024-05-10T21:30:08.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
792024-05-10T21:30:08.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
802024-05-10T21:30:08.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
812024-05-10T21:30:08.885ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
822024-05-10T21:30:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
832024-05-10T21:30:08.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
842024-05-10T21:30:08.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
852024-05-10T21:30:08.889ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.499718ms iteration = 13
862024-05-10T21:30:08.941ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
872024-05-10T21:30:08.941ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
882024-05-10T21:30:08.941ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "164000ed-e177-4edd-bd76-b946311cedeb", "content-length": "26", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
892024-05-10T21:30:08.941ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
902024-05-10T21:30:08.943ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
912024-05-10T21:30:08.945ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
922024-05-10T21:30:08.945ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
932024-05-10T21:30:08.945ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6bce53f5-3328-43e6-9d31-d9cbe5573ae7", "content-length": "26", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
942024-05-10T21:30:08.945ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
952024-05-10T21:30:08.948ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
962024-05-10T21:30:08.949ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
972024-05-10T21:30:08.949ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
982024-05-10T21:30:08.949ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "adc3a805-f240-4c61-a36b-37d883caa842", "content-length": "26", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
992024-05-10T21:30:08.949ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1002024-05-10T21:30:08.951ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
1012024-05-10T21:30:09.034ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1022024-05-10T21:30:09.034ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "099a18bd-8143-4cf3-ae8a-6c4b24e1fa85", "content-length": "1", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1032024-05-10T21:30:09.036ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1042024-05-10T21:30:09.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1052024-05-10T21:30:09.038ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 177.322087ms iteration = 13
1062024-05-10T21:30:09.121ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 260.655604ms iteration = 13
1072024-05-10T21:30:09.178ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1082024-05-10T21:30:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1092024-05-10T21:30:09.180ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1102024-05-10T21:30:09.180ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1112024-05-10T21:30:09.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1122024-05-10T21:30:09.187ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1132024-05-10T21:30:09.188ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4f5e92f1-5c7c-4afe-8afd-452b2e369ed4", "content-length": "121", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1142024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
1152024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
1162024-05-10T21:30:09.188ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1172024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
1182024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
1192024-05-10T21:30:09.188ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1202024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
1212024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
1222024-05-10T21:30:09.188ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
1232024-05-10T21:30:09.188ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1242024-05-10T21:30:09.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1252024-05-10T21:30:09.190ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1262024-05-10T21:30:09.190ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d0abec5c-8c43-4188-986d-baaf6b1f6d24", "content-length": "2", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1272024-05-10T21:30:09.190ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1282024-05-10T21:30:09.190ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2105b6e2-e3b6-4606-abaf-272439cb20f3", "content-length": "51", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1292024-05-10T21:30:09.190ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
1302024-05-10T21:30:09.190ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1312024-05-10T21:30:09.237ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "07a3870a-7c3c-471a-87d7-b6b3bef35eb2", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1322024-05-10T21:30:09.237ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
1332024-05-10T21:30:09.237ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1342024-05-10T21:30:09.238ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a5aaa677-5ffe-4517-96d2-14ca6ef9fe92", "content-length": "467", "date": "Fri, 10 May 2024 21:30:08 GMT"} })
1352024-05-10T21:30:09.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1362024-05-10T21:30:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1372024-05-10T21:30:09.241ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1382024-05-10T21:30:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1392024-05-10T21:30:09.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1402024-05-10T21:30:09.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1412024-05-10T21:30:09.246ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1422024-05-10T21:30:09.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1432024-05-10T21:30:09.253ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 392.391796ms iteration = 13
1442024-05-10T21:30:09.589ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 988 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 0938db9f-5170-461c-a465-fc6d8507cc6d response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
1452024-05-10T21:30:18.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
1462024-05-10T21:30:18.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1472024-05-10T21:30:18.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1482024-05-10T21:30:18.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1492024-05-10T21:30:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1502024-05-10T21:30:18.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1512024-05-10T21:30:18.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1522024-05-10T21:30:18.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1532024-05-10T21:30:18.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1542024-05-10T21:30:18.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1552024-05-10T21:30:18.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
1562024-05-10T21:30:18.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1572024-05-10T21:30:18.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1582024-05-10T21:30:18.877ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
1592024-05-10T21:30:18.877ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.862054ms iteration = 36
1602024-05-10T21:30:19.589ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 166 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = bdbc99b5-094d-4a0e-bb9b-3d24ecaefdb8 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
1612024-05-10T21:30:28.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
1622024-05-10T21:30:28.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1632024-05-10T21:30:28.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1642024-05-10T21:30:28.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1652024-05-10T21:30:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1662024-05-10T21:30:28.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1672024-05-10T21:30:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1682024-05-10T21:30:28.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1692024-05-10T21:30:28.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1702024-05-10T21:30:28.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1712024-05-10T21:30:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
1722024-05-10T21:30:28.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1732024-05-10T21:30:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1742024-05-10T21:30:28.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
1752024-05-10T21:30:28.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.844951ms iteration = 37
1762024-05-10T21:30:29.459ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1:0]:54912
1772024-05-10T21:30:29.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:54912 req_id = 1b1abcb6-76c4-4d69-bece-928830835543 roles = RoleSet { roles: {} } uri = /metrics/producers
1782024-05-10T21:30:29.460ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:54912 req_id = 1b1abcb6-76c4-4d69-bece-928830835543 resource = Database result = Ok(()) uri = /metrics/producers
1792024-05-10T21:30:29.495ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 95738557-5fbe-4760-a2f5-e295833eb800
1802024-05-10T21:30:29.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:54912 req_id = 1b1abcb6-76c4-4d69-bece-928830835543 roles = RoleSet { roles: {} } uri = /metrics/producers
1812024-05-10T21:30:29.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:54912 req_id = 1b1abcb6-76c4-4d69-bece-928830835543 resource = Database result = Ok(()) uri = /metrics/producers
1822024-05-10T21:30:29.500ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 uri = http://[fd00:1122:3344:101::d]:12223/producers
1832024-05-10T21:30:29.501ZDEBGnexus: client response oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ed6824cc-c466-491b-8a41-397770253b3b", "date": "Fri, 10 May 2024 21:30:29 GMT"} })
1842024-05-10T21:30:29.501ZINFOnexus: assigned collector to new producer collector_id = 95738557-5fbe-4760-a2f5-e295833eb800 file = nexus/src/app/oximeter.rs:125 producer_id = ad3c3a26-88d8-4e2a-8352-8fd346279818
1852024-05-10T21:30:29.504ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 43994 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:54912 req_id = 1b1abcb6-76c4-4d69-bece-928830835543 response_code = 201 uri = /metrics/producers
1862024-05-10T21:30:29.590ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 983 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 639bf934-4636-4c48-baf2-e4b0ae297781 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
1872024-05-10T21:30:38.858ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
1882024-05-10T21:30:38.858ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1892024-05-10T21:30:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1902024-05-10T21:30:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1912024-05-10T21:30:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1922024-05-10T21:30:38.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1932024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1942024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
1952024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
1962024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1972024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
1982024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
1992024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2002024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
2012024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2022024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
2032024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2042024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
2052024-05-10T21:30:38.860ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
2062024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
2072024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.71\u{b5}s iteration = 9
2082024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
2092024-05-10T21:30:38.860ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
2102024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.78\u{b5}s iteration = 8
2112024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
2122024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
2132024-05-10T21:30:38.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2142024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2152024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
2162024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
2172024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
2182024-05-10T21:30:38.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
2192024-05-10T21:30:38.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
2202024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2212024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.951\u{b5}s iteration = 14
2222024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2232024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
2242024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
2252024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2262024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2272024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2282024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2292024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2302024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2312024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2322024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2332024-05-10T21:30:38.860ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 638212748 } })
2342024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2352024-05-10T21:30:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2362024-05-10T21:30:38.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2372024-05-10T21:30:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2382024-05-10T21:30:38.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
2392024-05-10T21:30:38.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2402024-05-10T21:30:38.862ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 639493825 } })
2412024-05-10T21:30:38.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2422024-05-10T21:30:38.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2432024-05-10T21:30:38.862ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2442024-05-10T21:30:38.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2452024-05-10T21:30:38.863ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
2462024-05-10T21:30:38.863ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.235848ms iteration = 8
2472024-05-10T21:30:38.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2482024-05-10T21:30:38.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2492024-05-10T21:30:38.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
2502024-05-10T21:30:38.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.552665ms iteration = 14
2512024-05-10T21:30:38.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2522024-05-10T21:30:38.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2532024-05-10T21:30:38.864ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
2542024-05-10T21:30:38.864ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.580006ms iteration = 8
2552024-05-10T21:30:38.865ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
2562024-05-10T21:30:38.865ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
2572024-05-10T21:30:38.865ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
2582024-05-10T21:30:38.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2592024-05-10T21:30:38.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2602024-05-10T21:30:38.866ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 643745114 } })
2612024-05-10T21:30:38.866ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 643910597 } })
2622024-05-10T21:30:38.866ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
2632024-05-10T21:30:38.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2642024-05-10T21:30:38.867ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2652024-05-10T21:30:38.867ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2662024-05-10T21:30:38.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
2672024-05-10T21:30:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2682024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2692024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2702024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2712024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2722024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2732024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2742024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2752024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
2762024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2772024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2782024-05-10T21:30:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2792024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2802024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2812024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2822024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2832024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2842024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2852024-05-10T21:30:38.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2862024-05-10T21:30:38.871ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2872024-05-10T21:30:38.871ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
2882024-05-10T21:30:38.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2892024-05-10T21:30:38.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2902024-05-10T21:30:38.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2912024-05-10T21:30:38.872ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2922024-05-10T21:30:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2932024-05-10T21:30:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2942024-05-10T21:30:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2952024-05-10T21:30:38.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2962024-05-10T21:30:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2972024-05-10T21:30:38.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2982024-05-10T21:30:38.873ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1729, tv_nsec: 650917944 } })
2992024-05-10T21:30:38.873ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
3002024-05-10T21:30:38.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012024-05-10T21:30:38.874ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3022024-05-10T21:30:38.874ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3032024-05-10T21:30:38.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3042024-05-10T21:30:38.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3052024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
3062024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3072024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3082024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3092024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3102024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3112024-05-10T21:30:38.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3122024-05-10T21:30:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3132024-05-10T21:30:38.877ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3142024-05-10T21:30:38.877ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3152024-05-10T21:30:38.877ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3162024-05-10T21:30:38.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3172024-05-10T21:30:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3182024-05-10T21:30:38.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3192024-05-10T21:30:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3202024-05-10T21:30:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
3212024-05-10T21:30:38.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3222024-05-10T21:30:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3232024-05-10T21:30:38.878ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3242024-05-10T21:30:38.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3252024-05-10T21:30:38.879ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
3262024-05-10T21:30:38.879ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.515529ms iteration = 16
3272024-05-10T21:30:38.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3282024-05-10T21:30:38.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3292024-05-10T21:30:38.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3302024-05-10T21:30:38.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3312024-05-10T21:30:38.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3322024-05-10T21:30:38.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3332024-05-10T21:30:38.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3342024-05-10T21:30:38.883ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3352024-05-10T21:30:38.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3362024-05-10T21:30:38.884ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
3372024-05-10T21:30:38.884ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.965182ms iteration = 38
3382024-05-10T21:30:38.884ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
3392024-05-10T21:30:38.884ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3402024-05-10T21:30:38.884ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
3412024-05-10T21:30:38.884ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.19807ms iteration = 8
3422024-05-10T21:30:38.885ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
3432024-05-10T21:30:38.885ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
3442024-05-10T21:30:38.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3452024-05-10T21:30:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3462024-05-10T21:30:38.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3472024-05-10T21:30:38.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3482024-05-10T21:30:38.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3492024-05-10T21:30:38.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3502024-05-10T21:30:38.890ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
3512024-05-10T21:30:38.890ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3522024-05-10T21:30:38.890ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
3532024-05-10T21:30:38.890ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 30.507459ms iteration = 8
3542024-05-10T21:30:38.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3552024-05-10T21:30:38.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3562024-05-10T21:30:38.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3572024-05-10T21:30:38.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3582024-05-10T21:30:38.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3592024-05-10T21:30:38.896ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
3602024-05-10T21:30:38.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3612024-05-10T21:30:38.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3622024-05-10T21:30:38.897ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 37.235991ms iteration = 14
3632024-05-10T21:30:38.898ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
3642024-05-10T21:30:38.898ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 38.591689ms iteration = 8
3652024-05-10T21:30:39.010ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3662024-05-10T21:30:39.059ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3672024-05-10T21:30:39.070ZDEBGnexus (BackgroundTasks): client request SledAgent = 3ae06dcf-597c-4d18-b2d7-37c1f02096d7 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/ad3c3a26-88d8-4e2a-8352-8fd346279818/state
3682024-05-10T21:30:39.070ZDEBGnexus (BackgroundTasks): client response SledAgent = 3ae06dcf-597c-4d18-b2d7-37c1f02096d7 background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/ad3c3a26-88d8-4e2a-8352-8fd346279818/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2d9f02de-0018-436e-8b88-fb0ee18e8b62", "content-length": "313", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3692024-05-10T21:30:39.077ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 state = Running
3702024-05-10T21:30:39.079ZINFOnexus (BackgroundTasks): received new runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1979 instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 instance_state = InstanceRuntimeState { propolis_id: Some(6182ec66-497a-4e01-b9cc-9718ced2da39), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-05-10T21:30:21.845829526Z } propolis_id = 6182ec66-497a-4e01-b9cc-9718ced2da39 vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-05-10T21:30:29.534835444Z }
3712024-05-10T21:30:39.081ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3722024-05-10T21:30:39.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3732024-05-10T21:30:39.082ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
3742024-05-10T21:30:39.082ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3752024-05-10T21:30:39.082ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a118b26e-6faa-4917-bf17-350caf25f746", "content-length": "26", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3762024-05-10T21:30:39.082ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3772024-05-10T21:30:39.083ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
3782024-05-10T21:30:39.086ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3792024-05-10T21:30:39.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3802024-05-10T21:30:39.087ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3812024-05-10T21:30:39.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3822024-05-10T21:30:39.089ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3832024-05-10T21:30:39.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3842024-05-10T21:30:39.090ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3852024-05-10T21:30:39.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3862024-05-10T21:30:39.092ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {} }
3872024-05-10T21:30:39.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Database result = Ok(())
3882024-05-10T21:30:39.093ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3892024-05-10T21:30:39.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 47e71290-fa42-43c0-acb1-5a418035ba16, lookup_type: ById(47e71290-fa42-43c0-acb1-5a418035ba16) }, key: 45ce0783-2703-487d-910e-d5b22706e48e, lookup_type: ById(45ce0783-2703-487d-910e-d5b22706e48e) }, key: ad3c3a26-88d8-4e2a-8352-8fd346279818, lookup_type: ById(ad3c3a26-88d8-4e2a-8352-8fd346279818) } result = Ok(())
3902024-05-10T21:30:39.096ZDEBGnexus (BackgroundTasks): instance state generation already advanced, won't touch network config background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818
3912024-05-10T21:30:39.129ZINFOnexus (BackgroundTasks): instance and vmm updated by sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2112 instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 instance_updated = false propolis_id = 6182ec66-497a-4e01-b9cc-9718ced2da39 vmm_updated = false
3922024-05-10T21:30:39.129ZDEBGnexus (BackgroundTasks): update successful background_task = instance_watcher instance_id = ad3c3a26-88d8-4e2a-8352-8fd346279818 instance_updated = false state = Running vmm_updated = false
3932024-05-10T21:30:39.131ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1
3942024-05-10T21:30:39.131ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 269.135482ms iteration = 14
3952024-05-10T21:30:39.148ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
3962024-05-10T21:30:39.148ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3972024-05-10T21:30:39.149ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "787e286f-94dc-4dc4-bcd4-acd1d0ad0ffc", "content-length": "26", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
3982024-05-10T21:30:39.149ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3992024-05-10T21:30:39.151ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4002024-05-10T21:30:39.168ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
4012024-05-10T21:30:39.168ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4022024-05-10T21:30:39.168ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4491f2b3-d059-4cfb-8f15-7368c2d641b2", "content-length": "26", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4032024-05-10T21:30:39.168ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4042024-05-10T21:30:39.169ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4052024-05-10T21:30:39.213ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4062024-05-10T21:30:39.214ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5bcd1657-2e2f-4548-a79b-915d7c56f42b", "date": "Fri, 10 May 2024 21:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4072024-05-10T21:30:39.214ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4082024-05-10T21:30:39.215ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b4175442-9a6b-4721-92ee-ef2e55bb5dd2", "date": "Fri, 10 May 2024 21:30:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4092024-05-10T21:30:39.215ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4102024-05-10T21:30:39.216ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
4112024-05-10T21:30:39.216ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 355.954102ms iteration = 10
4122024-05-10T21:30:39.227ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
4132024-05-10T21:30:39.388ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4142024-05-10T21:30:39.389ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c64754bf-6b71-4132-9cb8-2044ccc06e93", "content-length": "1", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4152024-05-10T21:30:39.390ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4162024-05-10T21:30:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4172024-05-10T21:30:39.392ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 532.016473ms iteration = 14
4182024-05-10T21:30:39.400ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 540.126182ms iteration = 14
4192024-05-10T21:30:39.439ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
4202024-05-10T21:30:39.439ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "06326200-e925-4317-86e8-a6a2e980afa0", "date": "Fri, 10 May 2024 21:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4212024-05-10T21:30:39.439ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4222024-05-10T21:30:39.441ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d2e30225-70d3-4bc8-9862-e1083811e3a8", "date": "Fri, 10 May 2024 21:30:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4232024-05-10T21:30:39.441ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4242024-05-10T21:30:39.443ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a5f18723-fb95-49aa-b568-aa30fffbc56f", "date": "Fri, 10 May 2024 21:30:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4252024-05-10T21:30:39.443ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4262024-05-10T21:30:39.444ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
4272024-05-10T21:30:39.444ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 584.793708ms iteration = 10
4282024-05-10T21:30:39.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4292024-05-10T21:30:39.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4302024-05-10T21:30:39.486ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4312024-05-10T21:30:39.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4322024-05-10T21:30:39.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4332024-05-10T21:30:39.493ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4342024-05-10T21:30:39.493ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5023eef8-0a62-4886-b25f-75b206a2256b", "content-length": "121", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4352024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4362024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
4372024-05-10T21:30:39.493ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4382024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
4392024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
4402024-05-10T21:30:39.493ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4412024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
4422024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
4432024-05-10T21:30:39.493ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
4442024-05-10T21:30:39.493ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4452024-05-10T21:30:39.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4462024-05-10T21:30:39.495ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4472024-05-10T21:30:39.495ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bae347ad-d2c3-4e7b-92ff-4e4b63b0ca0e", "content-length": "2", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4482024-05-10T21:30:39.495ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4492024-05-10T21:30:39.495ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "27ffa67c-5cda-40b9-bd6a-69840d9719f9", "content-length": "51", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4502024-05-10T21:30:39.495ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
4512024-05-10T21:30:39.495ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4522024-05-10T21:30:39.540ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "88aaf4f3-55a2-4041-b06e-f3ece80191a3", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4532024-05-10T21:30:39.540ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
4542024-05-10T21:30:39.540ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4552024-05-10T21:30:39.544ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "629b40aa-9d4f-475e-a148-156ad60b14b8", "content-length": "467", "date": "Fri, 10 May 2024 21:30:38 GMT"} })
4562024-05-10T21:30:39.544ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4572024-05-10T21:30:39.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4582024-05-10T21:30:39.547ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4592024-05-10T21:30:39.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4602024-05-10T21:30:39.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4612024-05-10T21:30:39.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4622024-05-10T21:30:39.551ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4632024-05-10T21:30:39.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4642024-05-10T21:30:39.557ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 697.126923ms iteration = 14
4652024-05-10T21:30:39.591ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 2387 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 23b21d89-3334-4463-a101-f41494b7c15b response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
4662024-05-10T21:30:39.591ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
4672024-05-10T21:30:39.591ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
4682024-05-10T21:30:39.622ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
4692024-05-10T21:30:39.622ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:42962
4702024-05-10T21:30:39.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42962 req_id = a43d2aac-0508-4225-abcd-353dd03dc725 roles = RoleSet { roles: {} } uri = /metrics/producers
4712024-05-10T21:30:39.622ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42962 req_id = a43d2aac-0508-4225-abcd-353dd03dc725 resource = Database result = Ok(()) uri = /metrics/producers
4722024-05-10T21:30:39.654ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 95738557-5fbe-4760-a2f5-e295833eb800
4732024-05-10T21:30:39.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42962 req_id = a43d2aac-0508-4225-abcd-353dd03dc725 roles = RoleSet { roles: {} } uri = /metrics/producers
4742024-05-10T21:30:39.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42962 req_id = a43d2aac-0508-4225-abcd-353dd03dc725 resource = Database result = Ok(()) uri = /metrics/producers
4752024-05-10T21:30:39.680ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 uri = http://[fd00:1122:3344:101::d]:12223/producers
4762024-05-10T21:30:39.680ZDEBGnexus: client response oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "48d0aca0-f9af-410a-b30f-ee1d4ad33baa", "date": "Fri, 10 May 2024 21:30:39 GMT"} })
4772024-05-10T21:30:39.680ZINFOnexus: assigned collector to new producer collector_id = 95738557-5fbe-4760-a2f5-e295833eb800 file = nexus/src/app/oximeter.rs:125 producer_id = dd46fcb1-c07b-4880-9112-3895f2df61ed
4782024-05-10T21:30:39.682ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 60722 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:42962 req_id = a43d2aac-0508-4225-abcd-353dd03dc725 response_code = 201 uri = /metrics/producers
4792024-05-10T21:30:39.683ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "a43d2aac-0508-4225-abcd-353dd03dc725", "content-length": "41", "date": "Fri, 10 May 2024 21:30:39 GMT"} })
4802024-05-10T21:30:39.684ZDEBGnexus (producer-registration-task): registered with nexus successfully
4812024-05-10T21:30:39.684ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
4822024-05-10T21:30:48.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
4832024-05-10T21:30:48.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4842024-05-10T21:30:48.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4852024-05-10T21:30:48.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4862024-05-10T21:30:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4872024-05-10T21:30:48.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4882024-05-10T21:30:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4892024-05-10T21:30:48.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4902024-05-10T21:30:48.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4912024-05-10T21:30:48.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4922024-05-10T21:30:48.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
4932024-05-10T21:30:48.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4942024-05-10T21:30:48.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4952024-05-10T21:30:48.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
4962024-05-10T21:30:48.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.77784ms iteration = 39
4972024-05-10T21:30:49.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 243 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = aa1f5e61-0686-441b-9e8f-4c0dd246debb response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
4982024-05-10T21:30:58.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
4992024-05-10T21:30:58.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5002024-05-10T21:30:58.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5012024-05-10T21:30:58.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5022024-05-10T21:30:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5032024-05-10T21:30:58.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5042024-05-10T21:30:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5052024-05-10T21:30:58.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5062024-05-10T21:30:58.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5072024-05-10T21:30:58.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5082024-05-10T21:30:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
5092024-05-10T21:30:58.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5102024-05-10T21:30:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5112024-05-10T21:30:58.876ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
5122024-05-10T21:30:58.876ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.330482ms iteration = 40
5132024-05-10T21:30:59.681ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 218 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 16c03f68-75b0-450b-9d39-39b0166cf0a2 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
5142024-05-10T21:31:08.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
5152024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
5162024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5172024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5182024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
5192024-05-10T21:31:08.861ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5202024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
5212024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
5222024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
5232024-05-10T21:31:08.861ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5242024-05-10T21:31:08.861ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5252024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5262024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.17\u{b5}s iteration = 15
5272024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5282024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
5292024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
5302024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5312024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5322024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
5332024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5342024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5352024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5362024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5372024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5382024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5392024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5402024-05-10T21:31:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5412024-05-10T21:31:08.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5422024-05-10T21:31:08.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5432024-05-10T21:31:08.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5442024-05-10T21:31:08.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5452024-05-10T21:31:08.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
5462024-05-10T21:31:08.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.758558ms iteration = 15
5472024-05-10T21:31:08.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5482024-05-10T21:31:08.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1759, tv_nsec: 642661927 } })
5492024-05-10T21:31:08.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1759, tv_nsec: 642783060 } })
5502024-05-10T21:31:08.865ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
5512024-05-10T21:31:08.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5522024-05-10T21:31:08.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5532024-05-10T21:31:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5542024-05-10T21:31:08.866ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
5552024-05-10T21:31:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5562024-05-10T21:31:08.866ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5572024-05-10T21:31:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5582024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5592024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5602024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5612024-05-10T21:31:08.867ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1759, tv_nsec: 645234061 } })
5622024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5632024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5642024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5652024-05-10T21:31:08.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5662024-05-10T21:31:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5672024-05-10T21:31:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5682024-05-10T21:31:08.868ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
5692024-05-10T21:31:08.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5702024-05-10T21:31:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5712024-05-10T21:31:08.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5722024-05-10T21:31:08.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
5732024-05-10T21:31:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5742024-05-10T21:31:08.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5752024-05-10T21:31:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5762024-05-10T21:31:08.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5772024-05-10T21:31:08.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782024-05-10T21:31:08.871ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
5792024-05-10T21:31:08.872ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.97361ms iteration = 15
5802024-05-10T21:31:08.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5812024-05-10T21:31:08.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5822024-05-10T21:31:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5832024-05-10T21:31:08.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
5842024-05-10T21:31:08.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5852024-05-10T21:31:08.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5862024-05-10T21:31:08.879ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
5872024-05-10T21:31:08.879ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.409416ms iteration = 41
5882024-05-10T21:31:08.880ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5892024-05-10T21:31:08.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5902024-05-10T21:31:08.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5912024-05-10T21:31:08.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5922024-05-10T21:31:08.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5932024-05-10T21:31:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
5942024-05-10T21:31:08.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5952024-05-10T21:31:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5962024-05-10T21:31:08.888ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.726731ms iteration = 15
5972024-05-10T21:31:08.900ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
5982024-05-10T21:31:08.900ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 39.57906ms iteration = 17
5992024-05-10T21:31:08.914ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
6002024-05-10T21:31:08.914ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
6012024-05-10T21:31:08.914ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6022024-05-10T21:31:08.914ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6032024-05-10T21:31:08.915ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c97239e9-f7a0-48d1-a6b8-25645f7f2f31", "content-length": "26", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6042024-05-10T21:31:08.915ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6052024-05-10T21:31:08.916ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dbee3481-395e-4287-a767-6e15c8217fc7", "content-length": "26", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6062024-05-10T21:31:08.916ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6072024-05-10T21:31:08.917ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
6082024-05-10T21:31:08.918ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6092024-05-10T21:31:08.918ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
6102024-05-10T21:31:08.918ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6112024-05-10T21:31:08.919ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3ab88525-07a5-4d60-817a-3acaf73c5acb", "content-length": "26", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6122024-05-10T21:31:08.919ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6132024-05-10T21:31:08.921ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6142024-05-10T21:31:09.006ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
6152024-05-10T21:31:09.007ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d123b246-afa1-48d2-84e2-c48af79729e3", "content-length": "2", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6162024-05-10T21:31:09.008ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6172024-05-10T21:31:09.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6182024-05-10T21:31:09.010ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 149.203867ms iteration = 15
6192024-05-10T21:31:09.011ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 150.644587ms iteration = 15
6202024-05-10T21:31:09.067ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6212024-05-10T21:31:09.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6222024-05-10T21:31:09.069ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6232024-05-10T21:31:09.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6242024-05-10T21:31:09.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6252024-05-10T21:31:09.076ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6262024-05-10T21:31:09.077ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b1b20213-e989-4d27-91e8-84e6a23f3eca", "content-length": "121", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6272024-05-10T21:31:09.077ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
6282024-05-10T21:31:09.077ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
6292024-05-10T21:31:09.078ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6302024-05-10T21:31:09.078ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
6312024-05-10T21:31:09.078ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
6322024-05-10T21:31:09.078ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6332024-05-10T21:31:09.078ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
6342024-05-10T21:31:09.078ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
6352024-05-10T21:31:09.078ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
6362024-05-10T21:31:09.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6372024-05-10T21:31:09.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6382024-05-10T21:31:09.079ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6392024-05-10T21:31:09.079ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5ef9cc40-b7c0-46e6-81b6-013a97c402e8", "content-length": "2", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6402024-05-10T21:31:09.079ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6412024-05-10T21:31:09.079ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d7cf9370-f4b9-4d53-bbd2-166063e7ab2f", "content-length": "51", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6422024-05-10T21:31:09.079ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
6432024-05-10T21:31:09.079ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6442024-05-10T21:31:09.124ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "43628e9a-75f6-4e62-8efc-083f0e58385e", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6452024-05-10T21:31:09.124ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
6462024-05-10T21:31:09.124ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6472024-05-10T21:31:09.127ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d5d08ab3-cf76-44b4-9bbe-2d3873bf9d7b", "content-length": "467", "date": "Fri, 10 May 2024 21:31:08 GMT"} })
6482024-05-10T21:31:09.127ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6492024-05-10T21:31:09.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6502024-05-10T21:31:09.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6512024-05-10T21:31:09.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6522024-05-10T21:31:09.132ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6532024-05-10T21:31:09.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6542024-05-10T21:31:09.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6552024-05-10T21:31:09.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6562024-05-10T21:31:09.140ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 279.410097ms iteration = 15
6572024-05-10T21:31:09.681ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 206 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = ec9044f2-2937-464a-8293-1386b053fd47 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
6582024-05-10T21:31:18.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
6592024-05-10T21:31:18.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6602024-05-10T21:31:18.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6612024-05-10T21:31:18.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6622024-05-10T21:31:18.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6632024-05-10T21:31:18.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6642024-05-10T21:31:18.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6652024-05-10T21:31:18.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6662024-05-10T21:31:18.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6672024-05-10T21:31:18.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6682024-05-10T21:31:18.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
6692024-05-10T21:31:18.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6702024-05-10T21:31:18.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6712024-05-10T21:31:18.874ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
6722024-05-10T21:31:18.874ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.418592ms iteration = 42
6732024-05-10T21:31:19.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 156 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 8f216a88-9a7e-474a-8793-66a0b5d58dd2 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
6742024-05-10T21:31:28.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
6752024-05-10T21:31:28.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6762024-05-10T21:31:28.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6772024-05-10T21:31:28.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782024-05-10T21:31:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6792024-05-10T21:31:28.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6802024-05-10T21:31:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6812024-05-10T21:31:28.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6822024-05-10T21:31:28.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6832024-05-10T21:31:28.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6842024-05-10T21:31:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
6852024-05-10T21:31:28.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6862024-05-10T21:31:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6872024-05-10T21:31:28.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
6882024-05-10T21:31:28.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.759029ms iteration = 43
6892024-05-10T21:31:29.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 211 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = fbfba184-6df8-45ce-8fd6-4476391530a6 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
6902024-05-10T21:31:38.858ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
6912024-05-10T21:31:38.858ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6922024-05-10T21:31:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6932024-05-10T21:31:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6942024-05-10T21:31:38.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6952024-05-10T21:31:38.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6962024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6972024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
6982024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
6992024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7002024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
7012024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
7022024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7032024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
7042024-05-10T21:31:38.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7052024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
7062024-05-10T21:31:38.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
7072024-05-10T21:31:38.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
7082024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.831\u{b5}s iteration = 16
7092024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
7102024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
7112024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7122024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7132024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
7142024-05-10T21:31:38.860ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
7152024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.86\u{b5}s iteration = 9
7162024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
7172024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
7182024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7192024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
7202024-05-10T21:31:38.860ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
7212024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.691\u{b5}s iteration = 10
7222024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
7232024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7242024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
7252024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
7262024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
7272024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7282024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7292024-05-10T21:31:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7302024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
7312024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7322024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7332024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7342024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7352024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7362024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7372024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7382024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7392024-05-10T21:31:38.861ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 638822324 } })
7402024-05-10T21:31:38.861ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 638845574 } })
7412024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7422024-05-10T21:31:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7432024-05-10T21:31:38.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7442024-05-10T21:31:38.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7452024-05-10T21:31:38.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462024-05-10T21:31:38.862ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7472024-05-10T21:31:38.863ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7482024-05-10T21:31:38.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
7492024-05-10T21:31:38.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.792958ms iteration = 16
7502024-05-10T21:31:38.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7512024-05-10T21:31:38.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7522024-05-10T21:31:38.864ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
7532024-05-10T21:31:38.864ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.270359ms iteration = 9
7542024-05-10T21:31:38.865ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
7552024-05-10T21:31:38.865ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.898223ms iteration = 9
7562024-05-10T21:31:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 643101983 } })
7572024-05-10T21:31:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 643128014 } })
7582024-05-10T21:31:38.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
7592024-05-10T21:31:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7602024-05-10T21:31:38.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
7612024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7622024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7632024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7642024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7652024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7662024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7672024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
7682024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
7692024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
7702024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7712024-05-10T21:31:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7722024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7732024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7742024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7752024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7762024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7772024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7782024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7792024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7802024-05-10T21:31:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7812024-05-10T21:31:38.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7822024-05-10T21:31:38.870ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
7832024-05-10T21:31:38.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7842024-05-10T21:31:38.870ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1789, tv_nsec: 648090878 } })
7852024-05-10T21:31:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7862024-05-10T21:31:38.870ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
7872024-05-10T21:31:38.871ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7882024-05-10T21:31:38.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7892024-05-10T21:31:38.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7902024-05-10T21:31:38.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7912024-05-10T21:31:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7922024-05-10T21:31:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7932024-05-10T21:31:38.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942024-05-10T21:31:38.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7952024-05-10T21:31:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7962024-05-10T21:31:38.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7972024-05-10T21:31:38.873ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7982024-05-10T21:31:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7992024-05-10T21:31:38.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8002024-05-10T21:31:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8012024-05-10T21:31:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8022024-05-10T21:31:38.876ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8032024-05-10T21:31:38.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8042024-05-10T21:31:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8052024-05-10T21:31:38.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8062024-05-10T21:31:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8072024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
8082024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8092024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8102024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8112024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8122024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8132024-05-10T21:31:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8142024-05-10T21:31:38.879ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
8152024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 18.410435ms iteration = 18
8162024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
8172024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
8182024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
8192024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8202024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
8212024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
8222024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8232024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8242024-05-10T21:31:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8252024-05-10T21:31:38.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8262024-05-10T21:31:38.880ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
8272024-05-10T21:31:38.880ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 19.54592ms iteration = 16
8282024-05-10T21:31:38.880ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8292024-05-10T21:31:38.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8302024-05-10T21:31:38.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8312024-05-10T21:31:38.882ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8322024-05-10T21:31:38.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8332024-05-10T21:31:38.884ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8342024-05-10T21:31:38.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8352024-05-10T21:31:38.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8362024-05-10T21:31:38.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8372024-05-10T21:31:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8382024-05-10T21:31:38.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8392024-05-10T21:31:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8402024-05-10T21:31:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8412024-05-10T21:31:38.890ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
8422024-05-10T21:31:38.890ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.431157ms iteration = 44
8432024-05-10T21:31:38.894ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
8442024-05-10T21:31:38.894ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
8452024-05-10T21:31:38.894ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
8462024-05-10T21:31:38.895ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 36.400903ms iteration = 9
8472024-05-10T21:31:38.895ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
8482024-05-10T21:31:38.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8492024-05-10T21:31:38.895ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
8502024-05-10T21:31:38.895ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8512024-05-10T21:31:38.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8522024-05-10T21:31:38.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8532024-05-10T21:31:38.897ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542024-05-10T21:31:38.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8552024-05-10T21:31:38.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8562024-05-10T21:31:38.899ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8572024-05-10T21:31:38.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8582024-05-10T21:31:38.900ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8592024-05-10T21:31:38.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8602024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
8612024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8622024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
8632024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
8642024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
8652024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 41.949369ms iteration = 9
8662024-05-10T21:31:38.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8672024-05-10T21:31:38.904ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 43.792389ms iteration = 16
8682024-05-10T21:31:38.924ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
8692024-05-10T21:31:38.924ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 64.060992ms iteration = 9
8702024-05-10T21:31:38.973ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
8712024-05-10T21:31:38.995ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
8722024-05-10T21:31:38.995ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8732024-05-10T21:31:38.996ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "24caad6e-f332-43b8-91b8-d2f3f4a3afa4", "content-length": "26", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8742024-05-10T21:31:38.996ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8752024-05-10T21:31:38.998ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8762024-05-10T21:31:39.000ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
8772024-05-10T21:31:39.000ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8782024-05-10T21:31:39.000ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "931e9183-f9ba-4e4f-a297-8a10984bbd69", "content-length": "26", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8792024-05-10T21:31:39.000ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8802024-05-10T21:31:39.002ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
8812024-05-10T21:31:39.113ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
8822024-05-10T21:31:39.113ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e3669baf-912a-495f-86b4-7af7904b8fbf", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8832024-05-10T21:31:39.113ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8842024-05-10T21:31:39.115ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "df0de801-396d-4216-a6b6-96e7aefd62fa", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8852024-05-10T21:31:39.115ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8862024-05-10T21:31:39.116ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
8872024-05-10T21:31:39.116ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 256.077528ms iteration = 11
8882024-05-10T21:31:39.119ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
8892024-05-10T21:31:39.124ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
8902024-05-10T21:31:39.124ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8912024-05-10T21:31:39.125ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "323d5439-253f-4eff-9608-b40823dc27fc", "content-length": "26", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
8922024-05-10T21:31:39.125ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8932024-05-10T21:31:39.128ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8942024-05-10T21:31:39.242ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
8952024-05-10T21:31:39.306ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 445.881056ms iteration = 16
8962024-05-10T21:31:39.345ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
8972024-05-10T21:31:39.346ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f8e6de74-1966-4636-b4b9-594d66f425d4", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8982024-05-10T21:31:39.346ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
8992024-05-10T21:31:39.347ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ff90c19b-1520-48da-9204-c74595e61e45", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9002024-05-10T21:31:39.347ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9012024-05-10T21:31:39.349ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "37be8af0-6545-477d-9bdb-8676d1a449c3", "date": "Fri, 10 May 2024 21:31:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9022024-05-10T21:31:39.349ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9032024-05-10T21:31:39.351ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
9042024-05-10T21:31:39.351ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 490.614515ms iteration = 11
9052024-05-10T21:31:39.385ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9062024-05-10T21:31:39.385ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "06c4bab3-30b1-4e64-ad0b-7c4333794892", "content-length": "2", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9072024-05-10T21:31:39.386ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9082024-05-10T21:31:39.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9092024-05-10T21:31:39.388ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 527.274893ms iteration = 16
9102024-05-10T21:31:39.432ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9112024-05-10T21:31:39.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9122024-05-10T21:31:39.434ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9132024-05-10T21:31:39.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9142024-05-10T21:31:39.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9152024-05-10T21:31:39.440ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9162024-05-10T21:31:39.440ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a79c8ccf-0c02-4b68-a372-3a4259569456", "content-length": "121", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9172024-05-10T21:31:39.440ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
9182024-05-10T21:31:39.440ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
9192024-05-10T21:31:39.441ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9202024-05-10T21:31:39.441ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
9212024-05-10T21:31:39.441ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
9222024-05-10T21:31:39.441ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9232024-05-10T21:31:39.441ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
9242024-05-10T21:31:39.441ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
9252024-05-10T21:31:39.441ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
9262024-05-10T21:31:39.441ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9272024-05-10T21:31:39.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9282024-05-10T21:31:39.443ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9292024-05-10T21:31:39.443ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d799948c-26a5-4534-8ee9-8741a3ec1f1e", "content-length": "2", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9302024-05-10T21:31:39.443ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9312024-05-10T21:31:39.443ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c049df07-ecf0-4c13-bb3d-258326e3df94", "content-length": "51", "date": "Fri, 10 May 2024 21:31:38 GMT"} })
9322024-05-10T21:31:39.443ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
9332024-05-10T21:31:39.443ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9342024-05-10T21:31:39.486ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e6a14977-7275-4027-a7cd-fbaa5ad61784", "date": "Fri, 10 May 2024 21:31:39 GMT"} })
9352024-05-10T21:31:39.486ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
9362024-05-10T21:31:39.486ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9372024-05-10T21:31:39.486ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f0abf19d-6905-4840-856a-d770e4e8f727", "content-length": "467", "date": "Fri, 10 May 2024 21:31:39 GMT"} })
9382024-05-10T21:31:39.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9392024-05-10T21:31:39.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9402024-05-10T21:31:39.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9412024-05-10T21:31:39.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9422024-05-10T21:31:39.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9432024-05-10T21:31:39.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9442024-05-10T21:31:39.493ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9452024-05-10T21:31:39.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9462024-05-10T21:31:39.500ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 640.133658ms iteration = 16
9472024-05-10T21:31:39.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 635 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = bdd55de0-1d82-4e9f-b419-91dae15fe6a0 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
9482024-05-10T21:31:48.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
9492024-05-10T21:31:48.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9502024-05-10T21:31:48.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9512024-05-10T21:31:48.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9522024-05-10T21:31:48.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9532024-05-10T21:31:48.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9542024-05-10T21:31:48.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9552024-05-10T21:31:48.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9562024-05-10T21:31:48.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9572024-05-10T21:31:48.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9582024-05-10T21:31:48.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
9592024-05-10T21:31:48.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9602024-05-10T21:31:48.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9612024-05-10T21:31:48.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
9622024-05-10T21:31:48.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.875973ms iteration = 45
9632024-05-10T21:31:49.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 228 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = f1e93994-9888-4034-a57a-2faebe57503d response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
9642024-05-10T21:31:58.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
9652024-05-10T21:31:58.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9662024-05-10T21:31:58.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9672024-05-10T21:31:58.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9682024-05-10T21:31:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9692024-05-10T21:31:58.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9702024-05-10T21:31:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9712024-05-10T21:31:58.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9722024-05-10T21:31:58.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9732024-05-10T21:31:58.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9742024-05-10T21:31:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
9752024-05-10T21:31:58.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9762024-05-10T21:31:58.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9772024-05-10T21:31:58.876ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
9782024-05-10T21:31:58.876ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.761631ms iteration = 46
9792024-05-10T21:31:59.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 221 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 2eb0f9a9-d5cb-4ba0-9f52-959f0b45b669 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
9802024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
9812024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9822024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
9832024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9842024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
9852024-05-10T21:32:08.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
9862024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
9872024-05-10T21:32:08.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
9882024-05-10T21:32:08.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
9892024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.55\u{b5}s iteration = 17
9902024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
9912024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
9922024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9932024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
9942024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9952024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9962024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
9972024-05-10T21:32:08.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9982024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9992024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
10002024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10012024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10022024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10032024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10042024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10052024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10062024-05-10T21:32:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10072024-05-10T21:32:08.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10082024-05-10T21:32:08.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10092024-05-10T21:32:08.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10102024-05-10T21:32:08.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
10112024-05-10T21:32:08.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.468831ms iteration = 17
10122024-05-10T21:32:08.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10132024-05-10T21:32:08.863ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1819, tv_nsec: 641013726 } })
10142024-05-10T21:32:08.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10152024-05-10T21:32:08.864ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
10162024-05-10T21:32:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10172024-05-10T21:32:08.866ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10182024-05-10T21:32:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10192024-05-10T21:32:08.866ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1819, tv_nsec: 644247564 } })
10202024-05-10T21:32:08.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
10212024-05-10T21:32:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10222024-05-10T21:32:08.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10232024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10242024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10252024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10262024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10272024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10282024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10292024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10302024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10312024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10322024-05-10T21:32:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10332024-05-10T21:32:08.869ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
10342024-05-10T21:32:08.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10352024-05-10T21:32:08.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10362024-05-10T21:32:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10372024-05-10T21:32:08.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10382024-05-10T21:32:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10392024-05-10T21:32:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10402024-05-10T21:32:08.871ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1819, tv_nsec: 648426632 } })
10412024-05-10T21:32:08.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10422024-05-10T21:32:08.871ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
10432024-05-10T21:32:08.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10442024-05-10T21:32:08.871ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
10452024-05-10T21:32:08.871ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.785316ms iteration = 17
10462024-05-10T21:32:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10472024-05-10T21:32:08.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10482024-05-10T21:32:08.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10492024-05-10T21:32:08.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
10502024-05-10T21:32:08.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10512024-05-10T21:32:08.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10522024-05-10T21:32:08.876ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
10532024-05-10T21:32:08.876ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 15.890613ms iteration = 19
10542024-05-10T21:32:08.879ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
10552024-05-10T21:32:08.879ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.586879ms iteration = 47
10562024-05-10T21:32:08.886ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10572024-05-10T21:32:08.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10582024-05-10T21:32:08.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10592024-05-10T21:32:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10602024-05-10T21:32:08.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10612024-05-10T21:32:08.891ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
10622024-05-10T21:32:08.891ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10632024-05-10T21:32:08.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10642024-05-10T21:32:08.892ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.94006ms iteration = 17
10652024-05-10T21:32:08.954ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
10662024-05-10T21:32:08.954ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10672024-05-10T21:32:08.954ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e7479d16-6b48-4229-9b0c-b2a307416345", "content-length": "26", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10682024-05-10T21:32:08.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10692024-05-10T21:32:08.956ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10702024-05-10T21:32:09.111ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
10712024-05-10T21:32:09.111ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10722024-05-10T21:32:09.112ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7a7f64f7-241e-4a68-ab20-ea300c02220a", "content-length": "26", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10732024-05-10T21:32:09.112ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10742024-05-10T21:32:09.114ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
10752024-05-10T21:32:09.116ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
10762024-05-10T21:32:09.116ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10772024-05-10T21:32:09.117ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2af8a8a8-6908-4988-ab15-6d6fa37ad858", "content-length": "26", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10782024-05-10T21:32:09.117ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10792024-05-10T21:32:09.118ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
10802024-05-10T21:32:09.209ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10812024-05-10T21:32:09.209ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "db0c01e4-ae34-43cd-b6a1-d50ca98ec2ee", "content-length": "2", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10822024-05-10T21:32:09.210ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10832024-05-10T21:32:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10842024-05-10T21:32:09.211ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 351.037689ms iteration = 17
10852024-05-10T21:32:09.212ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 351.728913ms iteration = 17
10862024-05-10T21:32:09.286ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10872024-05-10T21:32:09.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10882024-05-10T21:32:09.288ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10892024-05-10T21:32:09.288ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10902024-05-10T21:32:09.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10912024-05-10T21:32:09.295ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
10922024-05-10T21:32:09.295ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "439fccd6-4b7f-4862-83d9-5353e9fff88c", "content-length": "121", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
10932024-05-10T21:32:09.295ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
10942024-05-10T21:32:09.295ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
10952024-05-10T21:32:09.296ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10962024-05-10T21:32:09.296ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
10972024-05-10T21:32:09.296ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
10982024-05-10T21:32:09.296ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
10992024-05-10T21:32:09.296ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
11002024-05-10T21:32:09.296ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
11012024-05-10T21:32:09.296ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
11022024-05-10T21:32:09.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11032024-05-10T21:32:09.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11042024-05-10T21:32:09.297ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
11052024-05-10T21:32:09.297ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0be62daa-613f-4538-a897-69f1cb2afa22", "content-length": "2", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
11062024-05-10T21:32:09.297ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
11072024-05-10T21:32:09.297ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4b604a3d-3ffb-4eef-b19a-906b7f56d13e", "content-length": "51", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
11082024-05-10T21:32:09.297ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
11092024-05-10T21:32:09.297ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
11102024-05-10T21:32:09.339ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "70ef4b22-362f-4bc9-8fcd-37dcbee7d0cb", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
11112024-05-10T21:32:09.339ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
11122024-05-10T21:32:09.339ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
11132024-05-10T21:32:09.339ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7505f604-e7a1-499a-9283-5f8b6961706e", "content-length": "467", "date": "Fri, 10 May 2024 21:32:08 GMT"} })
11142024-05-10T21:32:09.339ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11152024-05-10T21:32:09.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11162024-05-10T21:32:09.342ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11172024-05-10T21:32:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11182024-05-10T21:32:09.345ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11192024-05-10T21:32:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11202024-05-10T21:32:09.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11212024-05-10T21:32:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11222024-05-10T21:32:09.353ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 492.514404ms iteration = 17
11232024-05-10T21:32:09.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 214 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 8b9f6566-cc69-4761-8091-6019b55f5b4c response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
11242024-05-10T21:32:18.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
11252024-05-10T21:32:18.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11262024-05-10T21:32:18.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11272024-05-10T21:32:18.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11282024-05-10T21:32:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11292024-05-10T21:32:18.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11302024-05-10T21:32:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11312024-05-10T21:32:18.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11322024-05-10T21:32:18.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11332024-05-10T21:32:18.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11342024-05-10T21:32:18.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
11352024-05-10T21:32:18.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11362024-05-10T21:32:18.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11372024-05-10T21:32:18.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
11382024-05-10T21:32:18.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.874801ms iteration = 48
11392024-05-10T21:32:19.681ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 157 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 7de96ac9-7eba-4846-a3bc-536d5e5d10e7 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
11402024-05-10T21:32:28.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
11412024-05-10T21:32:28.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11422024-05-10T21:32:28.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11432024-05-10T21:32:28.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11442024-05-10T21:32:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11452024-05-10T21:32:28.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11462024-05-10T21:32:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11472024-05-10T21:32:28.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11482024-05-10T21:32:28.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11492024-05-10T21:32:28.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11502024-05-10T21:32:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
11512024-05-10T21:32:28.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11522024-05-10T21:32:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11532024-05-10T21:32:28.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
11542024-05-10T21:32:28.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.478593ms iteration = 49
11552024-05-10T21:32:29.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 158 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = b6ea0642-0541-4039-9c25-5c74d6a1fa86 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
11562024-05-10T21:32:38.858ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
11572024-05-10T21:32:38.858ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
11582024-05-10T21:32:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11592024-05-10T21:32:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11602024-05-10T21:32:38.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11612024-05-10T21:32:38.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11622024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
11632024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
11642024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11652024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
11662024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11672024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
11682024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
11692024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11702024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11712024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
11722024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11732024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
11742024-05-10T21:32:38.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
11752024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
11762024-05-10T21:32:38.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
11772024-05-10T21:32:38.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
11782024-05-10T21:32:38.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
11792024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.21\u{b5}s iteration = 18
11802024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
11812024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
11822024-05-10T21:32:38.861ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
11832024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
11842024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
11852024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.511\u{b5}s iteration = 10
11862024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11872024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11882024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
11892024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
11902024-05-10T21:32:38.861ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
11912024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.1\u{b5}s iteration = 11
11922024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
11932024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11942024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
11952024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
11962024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
11972024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11982024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11992024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12002024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12012024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12022024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12032024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12042024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12052024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12062024-05-10T21:32:38.861ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 639017232 } })
12072024-05-10T21:32:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12082024-05-10T21:32:38.862ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12092024-05-10T21:32:38.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12102024-05-10T21:32:38.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12112024-05-10T21:32:38.862ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 640337270 } })
12122024-05-10T21:32:38.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12132024-05-10T21:32:38.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12142024-05-10T21:32:38.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
12152024-05-10T21:32:38.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.86126ms iteration = 18
12162024-05-10T21:32:38.863ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12172024-05-10T21:32:38.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12182024-05-10T21:32:38.864ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
12192024-05-10T21:32:38.864ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.457933ms iteration = 10
12202024-05-10T21:32:38.864ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12212024-05-10T21:32:38.865ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
12222024-05-10T21:32:38.865ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.737149ms iteration = 10
12232024-05-10T21:32:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 643202770 } })
12242024-05-10T21:32:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 643210330 } })
12252024-05-10T21:32:38.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12262024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
12272024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
12282024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
12292024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12302024-05-10T21:32:38.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
12312024-05-10T21:32:38.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
12322024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12332024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
12342024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12352024-05-10T21:32:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12362024-05-10T21:32:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12372024-05-10T21:32:38.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12382024-05-10T21:32:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12392024-05-10T21:32:38.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12402024-05-10T21:32:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
12412024-05-10T21:32:38.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12422024-05-10T21:32:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12432024-05-10T21:32:38.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12442024-05-10T21:32:38.869ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12452024-05-10T21:32:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12462024-05-10T21:32:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12472024-05-10T21:32:38.869ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1849, tv_nsec: 647252684 } })
12482024-05-10T21:32:38.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12492024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12502024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12512024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12522024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12532024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12542024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12552024-05-10T21:32:38.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12562024-05-10T21:32:38.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12572024-05-10T21:32:38.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12582024-05-10T21:32:38.871ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
12592024-05-10T21:32:38.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
12602024-05-10T21:32:38.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12612024-05-10T21:32:38.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12622024-05-10T21:32:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12632024-05-10T21:32:38.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12642024-05-10T21:32:38.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12652024-05-10T21:32:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12662024-05-10T21:32:38.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12672024-05-10T21:32:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12682024-05-10T21:32:38.873ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12692024-05-10T21:32:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12702024-05-10T21:32:38.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12712024-05-10T21:32:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12722024-05-10T21:32:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12732024-05-10T21:32:38.874ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12742024-05-10T21:32:38.874ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12752024-05-10T21:32:38.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12762024-05-10T21:32:38.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12772024-05-10T21:32:38.874ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
12782024-05-10T21:32:38.874ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.412401ms iteration = 18
12792024-05-10T21:32:38.875ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
12802024-05-10T21:32:38.875ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12812024-05-10T21:32:38.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12822024-05-10T21:32:38.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12832024-05-10T21:32:38.876ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12842024-05-10T21:32:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
12852024-05-10T21:32:38.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12862024-05-10T21:32:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12872024-05-10T21:32:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12882024-05-10T21:32:38.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12892024-05-10T21:32:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12902024-05-10T21:32:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12912024-05-10T21:32:38.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12922024-05-10T21:32:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12932024-05-10T21:32:38.879ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12942024-05-10T21:32:38.879ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
12952024-05-10T21:32:38.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12962024-05-10T21:32:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12972024-05-10T21:32:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12982024-05-10T21:32:38.880ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12992024-05-10T21:32:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13002024-05-10T21:32:38.880ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13012024-05-10T21:32:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13022024-05-10T21:32:38.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13032024-05-10T21:32:38.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
13042024-05-10T21:32:38.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
13052024-05-10T21:32:38.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
13062024-05-10T21:32:38.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13072024-05-10T21:32:38.882ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
13082024-05-10T21:32:38.882ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.516031ms iteration = 50
13092024-05-10T21:32:38.882ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
13102024-05-10T21:32:38.882ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.846948ms iteration = 20
13112024-05-10T21:32:38.883ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
13122024-05-10T21:32:38.883ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
13132024-05-10T21:32:38.883ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
13142024-05-10T21:32:38.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.84685ms iteration = 10
13152024-05-10T21:32:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
13162024-05-10T21:32:38.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
13172024-05-10T21:32:38.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
13182024-05-10T21:32:38.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
13192024-05-10T21:32:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
13202024-05-10T21:32:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
13212024-05-10T21:32:38.886ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
13222024-05-10T21:32:38.886ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
13232024-05-10T21:32:38.886ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
13242024-05-10T21:32:38.886ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.329161ms iteration = 10
13252024-05-10T21:32:38.886ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13262024-05-10T21:32:38.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13272024-05-10T21:32:38.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13282024-05-10T21:32:38.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13292024-05-10T21:32:38.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13302024-05-10T21:32:38.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
13312024-05-10T21:32:38.892ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13322024-05-10T21:32:38.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13332024-05-10T21:32:38.894ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.38311ms iteration = 18
13342024-05-10T21:32:38.899ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
13352024-05-10T21:32:38.899ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 38.563008ms iteration = 10
13362024-05-10T21:32:38.983ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13372024-05-10T21:32:38.996ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
13382024-05-10T21:32:38.996ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13392024-05-10T21:32:38.996ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
13402024-05-10T21:32:38.997ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7fd8bafb-1771-495a-a14a-9e9718797737", "content-length": "26", "date": "Fri, 10 May 2024 21:32:38 GMT"} })
13412024-05-10T21:32:38.997ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13422024-05-10T21:32:38.999ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
13432024-05-10T21:32:39.000ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
13442024-05-10T21:32:39.000ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13452024-05-10T21:32:39.001ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "efe62c59-c188-4876-a709-fd850a685457", "content-length": "26", "date": "Fri, 10 May 2024 21:32:38 GMT"} })
13462024-05-10T21:32:39.001ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13472024-05-10T21:32:39.003ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13482024-05-10T21:32:39.012ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
13492024-05-10T21:32:39.012ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13502024-05-10T21:32:39.013ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5d5f9b84-0efb-4f35-be0f-867b125770fe", "content-length": "26", "date": "Fri, 10 May 2024 21:32:38 GMT"} })
13512024-05-10T21:32:39.013ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13522024-05-10T21:32:39.014ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13532024-05-10T21:32:39.123ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13542024-05-10T21:32:39.149ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13552024-05-10T21:32:39.149ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fa56f807-032d-4b0c-824a-c882bde31828", "date": "Fri, 10 May 2024 21:32:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13562024-05-10T21:32:39.149ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13572024-05-10T21:32:39.151ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1531fe87-d6ba-4a4c-9424-b098c3b526d1", "date": "Fri, 10 May 2024 21:32:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13582024-05-10T21:32:39.151ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13592024-05-10T21:32:39.153ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13602024-05-10T21:32:39.153ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 292.628945ms iteration = 12
13612024-05-10T21:32:39.160ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 299.404336ms iteration = 18
13622024-05-10T21:32:39.233ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13632024-05-10T21:32:39.233ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5dcb6747-dc81-4dd6-95ba-02966d48a5e5", "date": "Fri, 10 May 2024 21:32:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13642024-05-10T21:32:39.233ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13652024-05-10T21:32:39.235ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "94295304-4ab1-4f0c-99a3-57cd7a1e3292", "date": "Fri, 10 May 2024 21:32:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13662024-05-10T21:32:39.235ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13672024-05-10T21:32:39.237ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8b3fd248-441a-4b7b-89cc-d8c6da5485f1", "date": "Fri, 10 May 2024 21:32:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13682024-05-10T21:32:39.237ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13692024-05-10T21:32:39.239ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
13702024-05-10T21:32:39.239ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 378.832632ms iteration = 12
13712024-05-10T21:32:39.250ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13722024-05-10T21:32:39.251ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "893fdd0f-891e-4a57-b3d1-d2da35a2e770", "content-length": "2", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13732024-05-10T21:32:39.252ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13742024-05-10T21:32:39.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13752024-05-10T21:32:39.254ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 393.105041ms iteration = 18
13762024-05-10T21:32:39.307ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13772024-05-10T21:32:39.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13782024-05-10T21:32:39.309ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13792024-05-10T21:32:39.309ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13802024-05-10T21:32:39.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13812024-05-10T21:32:39.315ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13822024-05-10T21:32:39.315ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8edcadf2-844f-495c-93ac-4a6a50c540dd", "content-length": "121", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13832024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13842024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
13852024-05-10T21:32:39.315ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13862024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
13872024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
13882024-05-10T21:32:39.315ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13892024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
13902024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
13912024-05-10T21:32:39.315ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
13922024-05-10T21:32:39.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13932024-05-10T21:32:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13942024-05-10T21:32:39.317ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13952024-05-10T21:32:39.317ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "65c86a7e-7fb4-4d78-b5a4-fa35326e5f13", "content-length": "2", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13962024-05-10T21:32:39.317ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13972024-05-10T21:32:39.317ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ca2c187e-3a3d-40ca-8a64-b10681bb2402", "content-length": "51", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
13982024-05-10T21:32:39.317ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
13992024-05-10T21:32:39.317ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14002024-05-10T21:32:39.364ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "33ea01d8-ab22-4301-ac76-984ed719e826", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
14012024-05-10T21:32:39.364ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
14022024-05-10T21:32:39.364ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14032024-05-10T21:32:39.364ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "68084355-70e6-4e58-aa02-7e2f1036a786", "content-length": "467", "date": "Fri, 10 May 2024 21:32:39 GMT"} })
14042024-05-10T21:32:39.364ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14052024-05-10T21:32:39.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14062024-05-10T21:32:39.368ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14072024-05-10T21:32:39.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14082024-05-10T21:32:39.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14092024-05-10T21:32:39.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14102024-05-10T21:32:39.372ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14112024-05-10T21:32:39.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14122024-05-10T21:32:39.380ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 519.052881ms iteration = 18
14132024-05-10T21:32:39.681ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 132 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = cc48ebf1-39d1-47b5-9d6d-ab29db30f667 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
14142024-05-10T21:32:48.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
14152024-05-10T21:32:48.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14162024-05-10T21:32:48.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14172024-05-10T21:32:48.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14182024-05-10T21:32:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14192024-05-10T21:32:48.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14202024-05-10T21:32:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14212024-05-10T21:32:48.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14222024-05-10T21:32:48.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14232024-05-10T21:32:48.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14242024-05-10T21:32:48.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
14252024-05-10T21:32:48.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14262024-05-10T21:32:48.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14272024-05-10T21:32:48.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
14282024-05-10T21:32:48.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.25991ms iteration = 51
14292024-05-10T21:32:49.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 214 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 80e00a6c-464d-4f97-b50c-c6b7fcd97da3 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
14302024-05-10T21:32:58.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
14312024-05-10T21:32:58.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14322024-05-10T21:32:58.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14332024-05-10T21:32:58.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14342024-05-10T21:32:58.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14352024-05-10T21:32:58.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14362024-05-10T21:32:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14372024-05-10T21:32:58.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14382024-05-10T21:32:58.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14392024-05-10T21:32:58.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14402024-05-10T21:32:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
14412024-05-10T21:32:58.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14422024-05-10T21:32:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14432024-05-10T21:32:58.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
14442024-05-10T21:32:58.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.141987ms iteration = 52
14452024-05-10T21:32:59.683ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1146 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = ae8ddf13-3be0-431d-897a-c97f151d1407 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
14462024-05-10T21:33:08.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
14472024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
14482024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14492024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14502024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
14512024-05-10T21:33:08.861ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
14522024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
14532024-05-10T21:33:08.861ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
14542024-05-10T21:33:08.861ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
14552024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
14562024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
14572024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.361\u{b5}s iteration = 19
14582024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14592024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
14602024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
14612024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14622024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14632024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
14642024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14652024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14662024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14672024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14682024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14692024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14702024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14712024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14722024-05-10T21:33:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14732024-05-10T21:33:08.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742024-05-10T21:33:08.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14752024-05-10T21:33:08.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
14762024-05-10T21:33:08.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.564294ms iteration = 19
14772024-05-10T21:33:08.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14782024-05-10T21:33:08.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14792024-05-10T21:33:08.864ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14802024-05-10T21:33:08.864ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1879, tv_nsec: 641983151 } })
14812024-05-10T21:33:08.864ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1879, tv_nsec: 641989861 } })
14822024-05-10T21:33:08.865ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
14832024-05-10T21:33:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14842024-05-10T21:33:08.866ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14852024-05-10T21:33:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14862024-05-10T21:33:08.866ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
14872024-05-10T21:33:08.868ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
14882024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14892024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14902024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14912024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14922024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14932024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14942024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14952024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
14962024-05-10T21:33:08.868ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14972024-05-10T21:33:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14982024-05-10T21:33:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14992024-05-10T21:33:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15002024-05-10T21:33:08.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15012024-05-10T21:33:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15022024-05-10T21:33:08.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15032024-05-10T21:33:08.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15042024-05-10T21:33:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15052024-05-10T21:33:08.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15062024-05-10T21:33:08.871ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1879, tv_nsec: 648561189 } })
15072024-05-10T21:33:08.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15082024-05-10T21:33:08.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15092024-05-10T21:33:08.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
15102024-05-10T21:33:08.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15112024-05-10T21:33:08.872ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15122024-05-10T21:33:08.873ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
15132024-05-10T21:33:08.873ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.075764ms iteration = 19
15142024-05-10T21:33:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15152024-05-10T21:33:08.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
15162024-05-10T21:33:08.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15172024-05-10T21:33:08.875ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
15182024-05-10T21:33:08.875ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 14.573926ms iteration = 21
15192024-05-10T21:33:08.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15202024-05-10T21:33:08.880ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
15212024-05-10T21:33:08.880ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.252523ms iteration = 53
15222024-05-10T21:33:08.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15232024-05-10T21:33:08.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15242024-05-10T21:33:08.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15252024-05-10T21:33:08.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15262024-05-10T21:33:08.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15272024-05-10T21:33:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
15282024-05-10T21:33:08.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15292024-05-10T21:33:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15302024-05-10T21:33:08.888ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.447266ms iteration = 19
15312024-05-10T21:33:08.953ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
15322024-05-10T21:33:08.953ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15332024-05-10T21:33:08.954ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4773b916-a30c-4366-b49e-cc0f2de013b8", "content-length": "26", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15342024-05-10T21:33:08.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15352024-05-10T21:33:08.955ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15362024-05-10T21:33:08.955ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
15372024-05-10T21:33:08.955ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15382024-05-10T21:33:08.955ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d22f0fee-ce71-4767-bc9d-3128d3eb6c12", "content-length": "26", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15392024-05-10T21:33:08.955ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15402024-05-10T21:33:08.957ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15412024-05-10T21:33:08.965ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
15422024-05-10T21:33:08.965ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15432024-05-10T21:33:08.966ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1b746026-7e9f-4466-adf5-8c2ca75b0018", "content-length": "26", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15442024-05-10T21:33:08.966ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15452024-05-10T21:33:08.967ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
15462024-05-10T21:33:09.049ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15472024-05-10T21:33:09.050ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5eac215c-ff2e-4d5e-96df-681fff73f9ec", "content-length": "2", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15482024-05-10T21:33:09.050ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 189.172005ms iteration = 19
15492024-05-10T21:33:09.051ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15502024-05-10T21:33:09.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15512024-05-10T21:33:09.052ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 191.688408ms iteration = 19
15522024-05-10T21:33:09.112ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15532024-05-10T21:33:09.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15542024-05-10T21:33:09.114ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15552024-05-10T21:33:09.114ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15562024-05-10T21:33:09.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15572024-05-10T21:33:09.121ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15582024-05-10T21:33:09.121ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c28a52f9-6d82-44da-95dd-ab90e65b6fb7", "content-length": "121", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15592024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
15602024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
15612024-05-10T21:33:09.121ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15622024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
15632024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
15642024-05-10T21:33:09.121ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15652024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
15662024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
15672024-05-10T21:33:09.121ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
15682024-05-10T21:33:09.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15692024-05-10T21:33:09.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15702024-05-10T21:33:09.123ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15712024-05-10T21:33:09.123ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4b531eaa-e79c-45a6-a285-b50e074ce73a", "content-length": "2", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15722024-05-10T21:33:09.123ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15732024-05-10T21:33:09.123ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "548ba60c-f446-4628-ac18-e424c7edf5e3", "content-length": "51", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15742024-05-10T21:33:09.123ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
15752024-05-10T21:33:09.123ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15762024-05-10T21:33:09.169ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e14cedf8-6aca-4c22-8bc0-6aa351e99ad9", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15772024-05-10T21:33:09.169ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
15782024-05-10T21:33:09.169ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15792024-05-10T21:33:09.173ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b1bf83-22b5-4e82-936e-5a1a828714da", "content-length": "467", "date": "Fri, 10 May 2024 21:33:08 GMT"} })
15802024-05-10T21:33:09.173ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15812024-05-10T21:33:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15822024-05-10T21:33:09.176ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15832024-05-10T21:33:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15842024-05-10T21:33:09.178ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15852024-05-10T21:33:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15862024-05-10T21:33:09.180ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15872024-05-10T21:33:09.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15882024-05-10T21:33:09.186ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 325.853001ms iteration = 19
15892024-05-10T21:33:09.682ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 812 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 91f44894-74cc-4b16-9ab8-d2dabff6ae40 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
15902024-05-10T21:33:09.686ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
15912024-05-10T21:33:09.686ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
15922024-05-10T21:33:09.716ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
15932024-05-10T21:33:09.716ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:65386
15942024-05-10T21:33:09.716ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:65386 req_id = f17094f5-37af-48be-ab7a-2c13e4d9ae4d roles = RoleSet { roles: {} } uri = /metrics/producers
15952024-05-10T21:33:09.716ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:65386 req_id = f17094f5-37af-48be-ab7a-2c13e4d9ae4d resource = Database result = Ok(()) uri = /metrics/producers
15962024-05-10T21:33:09.748ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 95738557-5fbe-4760-a2f5-e295833eb800
15972024-05-10T21:33:09.748ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:65386 req_id = f17094f5-37af-48be-ab7a-2c13e4d9ae4d roles = RoleSet { roles: {} } uri = /metrics/producers
15982024-05-10T21:33:09.749ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:65386 req_id = f17094f5-37af-48be-ab7a-2c13e4d9ae4d resource = Database result = Ok(()) uri = /metrics/producers
15992024-05-10T21:33:09.777ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 uri = http://[fd00:1122:3344:101::d]:12223/producers
16002024-05-10T21:33:09.778ZDEBGnexus: client response oximeter-collector = 95738557-5fbe-4760-a2f5-e295833eb800 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "18903fbc-b514-4d6d-b056-0eb1f212a79f", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
16012024-05-10T21:33:09.778ZINFOnexus: assigned collector to new producer collector_id = 95738557-5fbe-4760-a2f5-e295833eb800 file = nexus/src/app/oximeter.rs:125 producer_id = dd46fcb1-c07b-4880-9112-3895f2df61ed
16022024-05-10T21:33:09.779ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 63327 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:65386 req_id = f17094f5-37af-48be-ab7a-2c13e4d9ae4d response_code = 201 uri = /metrics/producers
16032024-05-10T21:33:09.779ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "f17094f5-37af-48be-ab7a-2c13e4d9ae4d", "content-length": "41", "date": "Fri, 10 May 2024 21:33:09 GMT"} })
16042024-05-10T21:33:09.781ZDEBGnexus (producer-registration-task): registered with nexus successfully
16052024-05-10T21:33:09.781ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
16062024-05-10T21:33:18.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
16072024-05-10T21:33:18.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16082024-05-10T21:33:18.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16092024-05-10T21:33:18.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16102024-05-10T21:33:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16112024-05-10T21:33:18.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16122024-05-10T21:33:18.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16132024-05-10T21:33:18.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16142024-05-10T21:33:18.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16152024-05-10T21:33:18.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16162024-05-10T21:33:18.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
16172024-05-10T21:33:18.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16182024-05-10T21:33:18.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16192024-05-10T21:33:18.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
16202024-05-10T21:33:18.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.509405ms iteration = 54
16212024-05-10T21:33:19.779ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 242 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 3e1af0a1-8003-4e9d-a042-b540a6f947d6 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
16222024-05-10T21:33:28.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
16232024-05-10T21:33:28.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16242024-05-10T21:33:28.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16252024-05-10T21:33:28.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16262024-05-10T21:33:28.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16272024-05-10T21:33:28.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16282024-05-10T21:33:28.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16292024-05-10T21:33:28.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16302024-05-10T21:33:28.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16312024-05-10T21:33:28.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16322024-05-10T21:33:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
16332024-05-10T21:33:28.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16342024-05-10T21:33:28.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16352024-05-10T21:33:28.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
16362024-05-10T21:33:28.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.76042ms iteration = 55
16372024-05-10T21:33:29.779ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 159 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 4d9ae4e7-159b-487c-a083-66600a3faa9b response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
16382024-05-10T21:33:38.858ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
16392024-05-10T21:33:38.858ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16402024-05-10T21:33:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16412024-05-10T21:33:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16422024-05-10T21:33:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16432024-05-10T21:33:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16442024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
16452024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
16462024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
16472024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16482024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
16492024-05-10T21:33:38.860ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
16502024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.47\u{b5}s iteration = 12
16512024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
16522024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
16532024-05-10T21:33:38.860ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
16542024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.14\u{b5}s iteration = 11
16552024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
16562024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16572024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
16582024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
16592024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16602024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
16612024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16622024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16632024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16642024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
16652024-05-10T21:33:38.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
16662024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
16672024-05-10T21:33:38.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
16682024-05-10T21:33:38.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
16692024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
16702024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.07\u{b5}s iteration = 20
16712024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
16722024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
16732024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16742024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16752024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16762024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
16772024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16782024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16792024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16802024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16812024-05-10T21:33:38.860ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 638254751 } })
16822024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16832024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16842024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16852024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16862024-05-10T21:33:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16872024-05-10T21:33:38.860ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 638395454 } })
16882024-05-10T21:33:38.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
16892024-05-10T21:33:38.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16902024-05-10T21:33:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16912024-05-10T21:33:38.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16922024-05-10T21:33:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16932024-05-10T21:33:38.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16942024-05-10T21:33:38.862ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
16952024-05-10T21:33:38.862ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.354929ms iteration = 11
16962024-05-10T21:33:38.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16972024-05-10T21:33:38.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16982024-05-10T21:33:38.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
16992024-05-10T21:33:38.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.144346ms iteration = 20
17002024-05-10T21:33:38.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17012024-05-10T21:33:38.863ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022024-05-10T21:33:38.864ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17032024-05-10T21:33:38.865ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
17042024-05-10T21:33:38.865ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.111377ms iteration = 11
17052024-05-10T21:33:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 642933429 } })
17062024-05-10T21:33:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 642931189 } })
17072024-05-10T21:33:38.866ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
17082024-05-10T21:33:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17092024-05-10T21:33:38.867ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
17102024-05-10T21:33:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17112024-05-10T21:33:38.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17122024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17132024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
17142024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
17152024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
17162024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17172024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17182024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17192024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17202024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17212024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17222024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17232024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17242024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17252024-05-10T21:33:38.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17262024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17272024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17282024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17292024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17302024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17312024-05-10T21:33:38.869ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
17322024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17332024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17342024-05-10T21:33:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17352024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17362024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
17372024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
17382024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
17392024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17402024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17412024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17422024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17432024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17442024-05-10T21:33:38.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17452024-05-10T21:33:38.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17462024-05-10T21:33:38.871ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17472024-05-10T21:33:38.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17482024-05-10T21:33:38.871ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17492024-05-10T21:33:38.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17502024-05-10T21:33:38.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17512024-05-10T21:33:38.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1909, tv_nsec: 650074609 } })
17522024-05-10T21:33:38.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
17532024-05-10T21:33:38.875ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17542024-05-10T21:33:38.875ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17552024-05-10T21:33:38.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17562024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17572024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17582024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17592024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17602024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17612024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
17622024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17632024-05-10T21:33:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17642024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17652024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17662024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17672024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
17682024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17692024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17702024-05-10T21:33:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17712024-05-10T21:33:38.879ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
17722024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.184372ms iteration = 20
17732024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
17742024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
17752024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17762024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17772024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
17782024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17792024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
17802024-05-10T21:33:38.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17812024-05-10T21:33:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17822024-05-10T21:33:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17832024-05-10T21:33:38.880ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
17842024-05-10T21:33:38.880ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.206953ms iteration = 22
17852024-05-10T21:33:38.880ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17862024-05-10T21:33:38.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17872024-05-10T21:33:38.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17882024-05-10T21:33:38.883ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
17892024-05-10T21:33:38.883ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.279348ms iteration = 56
17902024-05-10T21:33:38.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17912024-05-10T21:33:38.884ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17922024-05-10T21:33:38.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17932024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17942024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17952024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17962024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17972024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17982024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17992024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18002024-05-10T21:33:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18012024-05-10T21:33:38.886ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
18022024-05-10T21:33:38.886ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
18032024-05-10T21:33:38.886ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
18042024-05-10T21:33:38.886ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 28.364844ms iteration = 11
18052024-05-10T21:33:38.886ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18062024-05-10T21:33:38.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18072024-05-10T21:33:38.887ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
18082024-05-10T21:33:38.887ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
18092024-05-10T21:33:38.887ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
18102024-05-10T21:33:38.887ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 27.21568ms iteration = 11
18112024-05-10T21:33:38.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18122024-05-10T21:33:38.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
18132024-05-10T21:33:38.890ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18142024-05-10T21:33:38.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18152024-05-10T21:33:38.892ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.974881ms iteration = 20
18162024-05-10T21:33:38.902ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
18172024-05-10T21:33:38.902ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 42.670104ms iteration = 11
18182024-05-10T21:33:39.000ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
18192024-05-10T21:33:39.007ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
18202024-05-10T21:33:39.018ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
18212024-05-10T21:33:39.018ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18222024-05-10T21:33:39.019ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d1252529-337b-4230-a2e2-9d4979603ec7", "content-length": "26", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18232024-05-10T21:33:39.019ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18242024-05-10T21:33:39.020ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
18252024-05-10T21:33:39.141ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
18262024-05-10T21:33:39.145ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
18272024-05-10T21:33:39.146ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fbc75112-54a0-49b6-ac33-f93ed846518f", "date": "Fri, 10 May 2024 21:33:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18282024-05-10T21:33:39.146ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18292024-05-10T21:33:39.148ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ab2974ba-6a22-4287-8019-19299b42ae25", "date": "Fri, 10 May 2024 21:33:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18302024-05-10T21:33:39.148ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18312024-05-10T21:33:39.150ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18322024-05-10T21:33:39.150ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 289.753784ms iteration = 13
18332024-05-10T21:33:39.156ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
18342024-05-10T21:33:39.156ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18352024-05-10T21:33:39.156ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1d518dca-91ff-4b74-9f65-42b1258c7b32", "content-length": "26", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18362024-05-10T21:33:39.156ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18372024-05-10T21:33:39.157ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
18382024-05-10T21:33:39.157ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18392024-05-10T21:33:39.157ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2aae8ded-a38e-4bb6-bf91-c7ac70c9a046", "content-length": "26", "date": "Fri, 10 May 2024 21:33:38 GMT"} })
18402024-05-10T21:33:39.157ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18412024-05-10T21:33:39.159ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18422024-05-10T21:33:39.159ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18432024-05-10T21:33:39.249ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
18442024-05-10T21:33:39.249ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "718a8f94-79a4-4eca-ab01-6007dd74ba0d", "date": "Fri, 10 May 2024 21:33:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18452024-05-10T21:33:39.249ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18462024-05-10T21:33:39.251ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4a2c35ee-85a8-4231-9b1b-796e8b289f18", "date": "Fri, 10 May 2024 21:33:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18472024-05-10T21:33:39.251ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18482024-05-10T21:33:39.253ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fddd1f48-9ac1-4f4f-8aba-123d634af2ae", "date": "Fri, 10 May 2024 21:33:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18492024-05-10T21:33:39.253ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18502024-05-10T21:33:39.255ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
18512024-05-10T21:33:39.255ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 395.013071ms iteration = 13
18522024-05-10T21:33:39.263ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 403.348705ms iteration = 20
18532024-05-10T21:33:39.264ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18542024-05-10T21:33:39.264ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3321d24c-b744-401c-9dc3-93411ab12792", "content-length": "2", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18552024-05-10T21:33:39.266ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18562024-05-10T21:33:39.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18572024-05-10T21:33:39.267ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 407.514763ms iteration = 20
18582024-05-10T21:33:39.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18592024-05-10T21:33:39.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18602024-05-10T21:33:39.295ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18612024-05-10T21:33:39.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18622024-05-10T21:33:39.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18632024-05-10T21:33:39.304ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
18642024-05-10T21:33:39.304ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f21f3dfe-1e3a-487c-8f6d-6a0ca09494e3", "content-length": "121", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18652024-05-10T21:33:39.304ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
18662024-05-10T21:33:39.304ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
18672024-05-10T21:33:39.309ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18682024-05-10T21:33:39.309ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
18692024-05-10T21:33:39.309ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
18702024-05-10T21:33:39.309ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18712024-05-10T21:33:39.309ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
18722024-05-10T21:33:39.309ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
18732024-05-10T21:33:39.309ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
18742024-05-10T21:33:39.309ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18752024-05-10T21:33:39.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18762024-05-10T21:33:39.310ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
18772024-05-10T21:33:39.310ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "873569bf-ff14-490c-88eb-559e1e4c803c", "content-length": "2", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18782024-05-10T21:33:39.310ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
18792024-05-10T21:33:39.310ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "398cbbb5-334f-4e4e-b024-69914ce85e3b", "content-length": "51", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18802024-05-10T21:33:39.310ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
18812024-05-10T21:33:39.310ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
18822024-05-10T21:33:39.377ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c003ccf2-018f-41db-907c-7f9a6ff727ea", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18832024-05-10T21:33:39.377ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
18842024-05-10T21:33:39.377ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
18852024-05-10T21:33:39.377ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9bc382f6-752e-480f-9f06-9e1bc869a586", "content-length": "467", "date": "Fri, 10 May 2024 21:33:39 GMT"} })
18862024-05-10T21:33:39.378ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18872024-05-10T21:33:39.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18882024-05-10T21:33:39.381ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18892024-05-10T21:33:39.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18902024-05-10T21:33:39.383ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18912024-05-10T21:33:39.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18922024-05-10T21:33:39.384ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18932024-05-10T21:33:39.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18942024-05-10T21:33:39.391ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 531.515252ms iteration = 20
18952024-05-10T21:33:39.778ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 153 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = caca1c50-f418-40ee-8118-731b732095ae response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
18962024-05-10T21:33:48.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
18972024-05-10T21:33:48.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18982024-05-10T21:33:48.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18992024-05-10T21:33:48.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19002024-05-10T21:33:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19012024-05-10T21:33:48.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19022024-05-10T21:33:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19032024-05-10T21:33:48.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19042024-05-10T21:33:48.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19052024-05-10T21:33:48.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19062024-05-10T21:33:48.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
19072024-05-10T21:33:48.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19082024-05-10T21:33:48.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19092024-05-10T21:33:48.874ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
19102024-05-10T21:33:48.874ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.77586ms iteration = 57
19112024-05-10T21:33:49.780ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1080 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 412bd4e7-1c4f-405b-8975-b5b7c62cc57f response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
19122024-05-10T21:33:58.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
19132024-05-10T21:33:58.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19142024-05-10T21:33:58.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19152024-05-10T21:33:58.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19162024-05-10T21:33:58.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19172024-05-10T21:33:58.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19182024-05-10T21:33:58.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19192024-05-10T21:33:58.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19202024-05-10T21:33:58.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19212024-05-10T21:33:58.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19222024-05-10T21:33:58.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
19232024-05-10T21:33:58.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19242024-05-10T21:33:58.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19252024-05-10T21:33:58.877ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
19262024-05-10T21:33:58.877ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.647649ms iteration = 58
19272024-05-10T21:33:59.779ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 156 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 74c64969-cea5-427f-b55d-6f846b4727d1 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
19282024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
19292024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
19302024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19312024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19322024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
19332024-05-10T21:34:08.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
19342024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
19352024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 21 reason = Timeout
19362024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
19372024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19382024-05-10T21:34:08.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
19392024-05-10T21:34:08.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
19402024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19412024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.231\u{b5}s iteration = 21
19422024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
19432024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
19442024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19452024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19462024-05-10T21:34:08.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19472024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
19482024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19492024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19502024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19512024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19522024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19532024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19542024-05-10T21:34:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19552024-05-10T21:34:08.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19562024-05-10T21:34:08.862ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19572024-05-10T21:34:08.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19582024-05-10T21:34:08.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19592024-05-10T21:34:08.863ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1939, tv_nsec: 641182780 } })
19602024-05-10T21:34:08.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
19612024-05-10T21:34:08.863ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19622024-05-10T21:34:08.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.254148ms iteration = 21
19632024-05-10T21:34:08.863ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1939, tv_nsec: 641381864 } })
19642024-05-10T21:34:08.864ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
19652024-05-10T21:34:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
19662024-05-10T21:34:08.866ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19672024-05-10T21:34:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
19682024-05-10T21:34:08.866ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19692024-05-10T21:34:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19702024-05-10T21:34:08.866ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
19712024-05-10T21:34:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19722024-05-10T21:34:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19732024-05-10T21:34:08.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19742024-05-10T21:34:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19752024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
19762024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19772024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19782024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19792024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19802024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19812024-05-10T21:34:08.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19822024-05-10T21:34:08.868ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1939, tv_nsec: 646396779 } })
19832024-05-10T21:34:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19842024-05-10T21:34:08.869ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
19852024-05-10T21:34:08.869ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19862024-05-10T21:34:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19872024-05-10T21:34:08.870ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
19882024-05-10T21:34:08.870ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 9.196813ms iteration = 21
19892024-05-10T21:34:08.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19902024-05-10T21:34:08.870ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
19912024-05-10T21:34:08.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19922024-05-10T21:34:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19932024-05-10T21:34:08.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19942024-05-10T21:34:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
19952024-05-10T21:34:08.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19962024-05-10T21:34:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19972024-05-10T21:34:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19982024-05-10T21:34:08.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19992024-05-10T21:34:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20002024-05-10T21:34:08.876ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
20012024-05-10T21:34:08.876ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 15.74546ms iteration = 23
20022024-05-10T21:34:08.878ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
20032024-05-10T21:34:08.878ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.772222ms iteration = 59
20042024-05-10T21:34:08.881ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20052024-05-10T21:34:08.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20062024-05-10T21:34:08.883ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20072024-05-10T21:34:08.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20082024-05-10T21:34:08.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20092024-05-10T21:34:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
20102024-05-10T21:34:08.887ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20112024-05-10T21:34:08.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20122024-05-10T21:34:08.888ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.100449ms iteration = 21
20132024-05-10T21:34:08.969ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
20142024-05-10T21:34:08.969ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20152024-05-10T21:34:08.970ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "967d7828-795f-410c-9624-beb91369c376", "content-length": "26", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20162024-05-10T21:34:08.970ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20172024-05-10T21:34:08.971ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20182024-05-10T21:34:09.049ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
20192024-05-10T21:34:09.049ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20202024-05-10T21:34:09.049ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ff8bc455-3831-48f3-aa63-325938100561", "content-length": "26", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20212024-05-10T21:34:09.049ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20222024-05-10T21:34:09.052ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20232024-05-10T21:34:09.064ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
20242024-05-10T21:34:09.064ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20252024-05-10T21:34:09.065ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "34645a22-94eb-4577-8cf3-4667739bdf04", "content-length": "26", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20262024-05-10T21:34:09.065ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20272024-05-10T21:34:09.067ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
20282024-05-10T21:34:09.071ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
20292024-05-10T21:34:09.071ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "236f8a41-bb69-4c21-ba16-98545046d18e", "content-length": "2", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20302024-05-10T21:34:09.072ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20312024-05-10T21:34:09.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20322024-05-10T21:34:09.074ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 213.616967ms iteration = 21
20332024-05-10T21:34:09.127ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 267.076629ms iteration = 21
20342024-05-10T21:34:09.188ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20352024-05-10T21:34:09.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20362024-05-10T21:34:09.190ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
20372024-05-10T21:34:09.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20382024-05-10T21:34:09.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20392024-05-10T21:34:09.196ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
20402024-05-10T21:34:09.196ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3b4c6591-ae56-4bc7-bbe9-adf7cf9a1226", "content-length": "121", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20412024-05-10T21:34:09.196ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
20422024-05-10T21:34:09.196ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
20432024-05-10T21:34:09.197ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
20442024-05-10T21:34:09.197ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
20452024-05-10T21:34:09.197ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
20462024-05-10T21:34:09.197ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
20472024-05-10T21:34:09.197ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
20482024-05-10T21:34:09.197ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
20492024-05-10T21:34:09.197ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
20502024-05-10T21:34:09.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20512024-05-10T21:34:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20522024-05-10T21:34:09.198ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
20532024-05-10T21:34:09.198ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bc29370c-7328-4780-aaec-a3a83c7853b4", "content-length": "2", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20542024-05-10T21:34:09.198ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
20552024-05-10T21:34:09.198ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "395573c3-99cc-4fbf-951c-f54c51dd289c", "content-length": "51", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20562024-05-10T21:34:09.198ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
20572024-05-10T21:34:09.198ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20582024-05-10T21:34:09.242ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4e0a12a5-2e2a-4b57-a792-3fef720b4316", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20592024-05-10T21:34:09.242ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
20602024-05-10T21:34:09.242ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20612024-05-10T21:34:09.243ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "89d5df69-756e-4867-8107-5cb6bba60757", "content-length": "467", "date": "Fri, 10 May 2024 21:34:08 GMT"} })
20622024-05-10T21:34:09.243ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20632024-05-10T21:34:09.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20642024-05-10T21:34:09.246ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20652024-05-10T21:34:09.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20662024-05-10T21:34:09.248ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20672024-05-10T21:34:09.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20682024-05-10T21:34:09.249ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20692024-05-10T21:34:09.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20702024-05-10T21:34:09.257ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 396.808988ms iteration = 21
20712024-05-10T21:34:09.778ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 244 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = a150e2f5-bda3-4ed1-b46c-e0add978807d response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
20722024-05-10T21:34:18.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
20732024-05-10T21:34:18.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20742024-05-10T21:34:18.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20752024-05-10T21:34:18.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20762024-05-10T21:34:18.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20772024-05-10T21:34:18.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20782024-05-10T21:34:18.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20792024-05-10T21:34:18.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20802024-05-10T21:34:18.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20812024-05-10T21:34:18.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20822024-05-10T21:34:18.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
20832024-05-10T21:34:18.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20842024-05-10T21:34:18.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20852024-05-10T21:34:18.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
20862024-05-10T21:34:18.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.25844ms iteration = 60
20872024-05-10T21:34:19.778ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 223 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 2eb49cf0-f9b1-4ff5-8f02-8e562995cfb1 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
20882024-05-10T21:34:28.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
20892024-05-10T21:34:28.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20902024-05-10T21:34:28.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20912024-05-10T21:34:28.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20922024-05-10T21:34:28.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20932024-05-10T21:34:28.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20942024-05-10T21:34:28.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20952024-05-10T21:34:28.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20962024-05-10T21:34:28.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20972024-05-10T21:34:28.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20982024-05-10T21:34:28.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
20992024-05-10T21:34:28.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21002024-05-10T21:34:28.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21012024-05-10T21:34:28.874ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
21022024-05-10T21:34:28.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.939993ms iteration = 61
21032024-05-10T21:34:29.778ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 236 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = c78834fe-4ede-419c-9731-49ef4497bf3d response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
21042024-05-10T21:34:38.857ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
21052024-05-10T21:34:38.858ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
21062024-05-10T21:34:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21072024-05-10T21:34:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21082024-05-10T21:34:38.858ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21092024-05-10T21:34:38.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21102024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21112024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
21122024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21132024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
21142024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
21152024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
21162024-05-10T21:34:38.860ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = 6cd98671-8b12-4197-ac01-c6571c567391
21172024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.98\u{b5}s iteration = 13
21182024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
21192024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Timeout
21202024-05-10T21:34:38.860ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
21212024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.33\u{b5}s iteration = 12
21222024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21232024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
21242024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
21252024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21262024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
21272024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
21282024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21292024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
21302024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
21312024-05-10T21:34:38.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
21322024-05-10T21:34:38.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
21332024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.99\u{b5}s iteration = 22
21342024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21352024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
21362024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
21372024-05-10T21:34:38.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
21382024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
21392024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
21402024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21412024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21422024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
21432024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
21442024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21452024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21462024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21472024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21482024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21492024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21502024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21512024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21522024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21532024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21542024-05-10T21:34:38.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21552024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
21562024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
21572024-05-10T21:34:38.861ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
21582024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21592024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21602024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21612024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21622024-05-10T21:34:38.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21632024-05-10T21:34:38.862ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7325b560-f7aa-4399-a6c6-a3322f42d759.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e03d4432-8b77-4406-904d-ebd71cc9739e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1969, tv_nsec: 639671126 } })
21642024-05-10T21:34:38.862ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("44b8a2e7-6078-49de-bb34-f1cd8770385e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8749ab70-ffc4-4a3c-a81f-43992c9390ce.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("e564d557-0b19-497f-baac-984106980347.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1969, tv_nsec: 639772298 } })
21652024-05-10T21:34:38.862ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21662024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21672024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21682024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21692024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21702024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21712024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21722024-05-10T21:34:38.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
21732024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.571564ms iteration = 22
21742024-05-10T21:34:38.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21752024-05-10T21:34:38.864ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
21762024-05-10T21:34:38.864ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.045895ms iteration = 12
21772024-05-10T21:34:38.864ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21782024-05-10T21:34:38.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21792024-05-10T21:34:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1969, tv_nsec: 642794651 } })
21802024-05-10T21:34:38.865ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1969, tv_nsec: 642949224 } })
21812024-05-10T21:34:38.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21822024-05-10T21:34:38.865ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21832024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21842024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
21852024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
21862024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
21872024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21882024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
21892024-05-10T21:34:38.866ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.67778ms iteration = 12
21902024-05-10T21:34:38.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21912024-05-10T21:34:38.867ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
21922024-05-10T21:34:38.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
21932024-05-10T21:34:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
21942024-05-10T21:34:38.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21952024-05-10T21:34:38.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21962024-05-10T21:34:38.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
21972024-05-10T21:34:38.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21982024-05-10T21:34:38.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21992024-05-10T21:34:38.868ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22002024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
22012024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
22022024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22032024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22042024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
22052024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22062024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22072024-05-10T21:34:38.869ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22082024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22092024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22102024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22112024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
22122024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22132024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22142024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22152024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22162024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22172024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22182024-05-10T21:34:38.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22192024-05-10T21:34:38.871ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22202024-05-10T21:34:38.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22212024-05-10T21:34:38.871ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
22222024-05-10T21:34:38.871ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22232024-05-10T21:34:38.871ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22242024-05-10T21:34:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22252024-05-10T21:34:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22262024-05-10T21:34:38.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22272024-05-10T21:34:38.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22282024-05-10T21:34:38.872ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1969, tv_nsec: 649830168 } })
22292024-05-10T21:34:38.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22302024-05-10T21:34:38.872ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
22312024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(e0db4117-15b7-4521-a093-70343c11356e, 0), (583c2104-c807-4b49-b29e-45ce2598a8c6, 0), (6aa2b03a-d3f4-4125-a603-2d3defe6e918, 0)]
22322024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
22332024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
22342024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
22352024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22362024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22372024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22382024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22392024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22402024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22412024-05-10T21:34:38.873ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22422024-05-10T21:34:38.874ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22432024-05-10T21:34:38.874ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1969, tv_nsec: 652553226 } })
22442024-05-10T21:34:38.875ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
22452024-05-10T21:34:38.875ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.881711ms iteration = 22
22462024-05-10T21:34:38.875ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22472024-05-10T21:34:38.875ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
22482024-05-10T21:34:38.875ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22492024-05-10T21:34:38.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22502024-05-10T21:34:38.877ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22512024-05-10T21:34:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22522024-05-10T21:34:38.877ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22532024-05-10T21:34:38.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22542024-05-10T21:34:38.877ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22552024-05-10T21:34:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22562024-05-10T21:34:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22572024-05-10T21:34:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
22582024-05-10T21:34:38.878ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22592024-05-10T21:34:38.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22602024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
22612024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22622024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22632024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22642024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22652024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22662024-05-10T21:34:38.879ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22672024-05-10T21:34:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22682024-05-10T21:34:38.880ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22692024-05-10T21:34:38.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22702024-05-10T21:34:38.880ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
22712024-05-10T21:34:38.880ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.592482ms iteration = 24
22722024-05-10T21:34:38.881ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22732024-05-10T21:34:38.881ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 0 version = 2
22742024-05-10T21:34:38.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22752024-05-10T21:34:38.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
22762024-05-10T21:34:38.881ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22772024-05-10T21:34:38.881ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22782024-05-10T21:34:38.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22792024-05-10T21:34:38.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22802024-05-10T21:34:38.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
22812024-05-10T21:34:38.882ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
22822024-05-10T21:34:38.883ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22832024-05-10T21:34:38.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
22842024-05-10T21:34:38.883ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22852024-05-10T21:34:38.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22862024-05-10T21:34:38.884ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
22872024-05-10T21:34:38.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22882024-05-10T21:34:38.884ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22892024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
22902024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22912024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fda79e3c-bcf6-4d27-a7c2-63d4c6089e8c dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
22922024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
22932024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-10 21:24:44.717873 UTC dns_group = internal generation = 1
22942024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.350823ms iteration = 12
22952024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
22962024-05-10T21:34:38.885ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.237729ms iteration = 62
22972024-05-10T21:34:38.886ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22982024-05-10T21:34:38.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22992024-05-10T21:34:38.886ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23002024-05-10T21:34:38.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23012024-05-10T21:34:38.888ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 18d3b658-affb-481c-b0ca-9e294961fba1 dns_zone_name = oxide.test found_so_far = 1 version = 2
23022024-05-10T21:34:38.888ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
23032024-05-10T21:34:38.888ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-10 21:24:45.322994 UTC dns_group = external generation = 2
23042024-05-10T21:34:38.888ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.273483ms iteration = 12
23052024-05-10T21:34:38.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
23062024-05-10T21:34:38.890ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23072024-05-10T21:34:38.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23082024-05-10T21:34:38.893ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23092024-05-10T21:34:38.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23102024-05-10T21:34:38.893ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23112024-05-10T21:34:38.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23122024-05-10T21:34:38.894ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23132024-05-10T21:34:38.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23142024-05-10T21:34:38.895ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23152024-05-10T21:34:38.895ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23162024-05-10T21:34:38.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23172024-05-10T21:34:38.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23182024-05-10T21:34:38.896ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23192024-05-10T21:34:38.896ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23202024-05-10T21:34:38.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23212024-05-10T21:34:38.898ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23222024-05-10T21:34:38.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23232024-05-10T21:34:38.899ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23242024-05-10T21:34:38.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23252024-05-10T21:34:38.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
23262024-05-10T21:34:38.899ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23272024-05-10T21:34:38.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23282024-05-10T21:34:38.900ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23292024-05-10T21:34:38.901ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 41.056721ms iteration = 22
23302024-05-10T21:34:38.902ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-10T21:24:38.821313Z, time_modified: 2024-05-10T21:24:38.821313Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 47e71290-fa42-43c0-acb1-5a418035ba16, db_silo: Silo { identity: SiloIdentity { id: 47e71290-fa42-43c0-acb1-5a418035ba16, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-10T21:24:45.274178Z, time_modified: 2024-05-10T21:24:45.274178Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "2c55031fbf3941fe05a22e171e8f0a44a60fb05b3f0b86ea9948f794454b7c6c" }] }) }
23312024-05-10T21:34:38.902ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 41.828607ms iteration = 12
23322024-05-10T21:34:38.904ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
23332024-05-10T21:34:38.904ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23342024-05-10T21:34:38.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23352024-05-10T21:34:38.905ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23362024-05-10T21:34:38.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23372024-05-10T21:34:38.907ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23382024-05-10T21:34:38.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23392024-05-10T21:34:38.908ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23402024-05-10T21:34:38.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23412024-05-10T21:34:38.910ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23422024-05-10T21:34:38.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23432024-05-10T21:34:38.911ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23442024-05-10T21:34:38.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23452024-05-10T21:34:38.912ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23462024-05-10T21:34:38.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23472024-05-10T21:34:38.913ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23482024-05-10T21:34:38.919ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23492024-05-10T21:34:38.919ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23502024-05-10T21:34:38.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23512024-05-10T21:34:38.920ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23522024-05-10T21:34:38.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23532024-05-10T21:34:38.921ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23542024-05-10T21:34:38.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23552024-05-10T21:34:38.925ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23562024-05-10T21:34:38.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
23572024-05-10T21:34:38.930ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23582024-05-10T21:34:38.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23592024-05-10T21:34:38.938ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23602024-05-10T21:34:38.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23612024-05-10T21:34:38.940ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23622024-05-10T21:34:38.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23632024-05-10T21:34:38.942ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23642024-05-10T21:34:38.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23652024-05-10T21:34:38.943ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23662024-05-10T21:34:38.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23672024-05-10T21:34:38.945ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23682024-05-10T21:34:38.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23692024-05-10T21:34:38.947ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23702024-05-10T21:34:38.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23712024-05-10T21:34:38.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23722024-05-10T21:34:38.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23732024-05-10T21:34:38.949ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23742024-05-10T21:34:38.954ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23752024-05-10T21:34:38.954ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23762024-05-10T21:34:38.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23772024-05-10T21:34:38.955ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23782024-05-10T21:34:38.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23792024-05-10T21:34:38.957ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23802024-05-10T21:34:38.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23812024-05-10T21:34:38.958ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23822024-05-10T21:34:38.962ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
23832024-05-10T21:34:38.962ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23842024-05-10T21:34:38.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23852024-05-10T21:34:38.968ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 476144d9-7af0-48ae-ae6d-f7b1547a9af0, kind: Service { id: dd46fcb1-c07b-4880-9112-3895f2df61ed }, name: Name("nexus-dd46fcb1-c07b-4880-9112-3895f2df61ed"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 174])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6768c786-a56a-479c-a1a1-b0566791d10c, kind: Service { id: 2fc7758a-cae3-4d9b-ade9-3a7691598581 }, name: Name("nexus-2fc7758a-cae3-4d9b-ade9-3a7691598581"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 253, 85])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a28a0581-77ae-4cf2-b9a8-382fafb05295, kind: Service { id: 7bb2217a-5bab-47ca-8e9e-29f166894592 }, name: Name("nexus-7bb2217a-5bab-47ca-8e9e-29f166894592"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 221])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 3076c844-8a4f-4e23-8343-ae2edafd59bc, kind: Service { id: e03d4432-8b77-4406-904d-ebd71cc9739e }, name: Name("external-dns-e03d4432-8b77-4406-904d-ebd71cc9739e"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 130, 3])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9f5600e8-2094-4425-bdd8-3005882ee50f, kind: Service { id: 7325b560-f7aa-4399-a6c6-a3322f42d759 }, name: Name("external-dns-7325b560-f7aa-4399-a6c6-a3322f42d759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 190])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
23862024-05-10T21:34:38.968ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23872024-05-10T21:34:38.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23882024-05-10T21:34:38.969ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23892024-05-10T21:34:38.972ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
23902024-05-10T21:34:38.972ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23912024-05-10T21:34:38.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
23922024-05-10T21:34:38.974ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
23932024-05-10T21:34:38.974ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3076c844-8a4f-4e23-8343-ae2edafd59bc, kind: Service { id: e03d4432-8b77-4406-904d-ebd71cc9739e }, name: Name("external-dns-e03d4432-8b77-4406-904d-ebd71cc9739e"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 130, 3])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9f5600e8-2094-4425-bdd8-3005882ee50f, kind: Service { id: 7325b560-f7aa-4399-a6c6-a3322f42d759 }, name: Name("external-dns-7325b560-f7aa-4399-a6c6-a3322f42d759"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 142, 190])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.1.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }] }, VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 476144d9-7af0-48ae-ae6d-f7b1547a9af0, kind: Service { id: dd46fcb1-c07b-4880-9112-3895f2df61ed }, name: Name("nexus-dd46fcb1-c07b-4880-9112-3895f2df61ed"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 174])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 6768c786-a56a-479c-a1a1-b0566791d10c, kind: Service { id: 2fc7758a-cae3-4d9b-ade9-3a7691598581 }, name: Name("nexus-2fc7758a-cae3-4d9b-ade9-3a7691598581"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 253, 85])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a28a0581-77ae-4cf2-b9a8-382fafb05295, kind: Service { id: 7bb2217a-5bab-47ca-8e9e-29f166894592 }, name: Name("nexus-7bb2217a-5bab-47ca-8e9e-29f166894592"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 255, 221])), subnet: V4(Ipv4Net(Ipv4Network { addr: 172.30.2.0, prefix: 24 })), vni: Vni(100), primary: true, slot: 0 }] }]
23942024-05-10T21:34:38.974ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
23952024-05-10T21:34:39.011ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
23962024-05-10T21:34:39.025ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, time_created: 2024-05-10T21:24:42.288783Z, time_modified: 2024-05-10T21:24:42.288783Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: 0, usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
23972024-05-10T21:34:39.025ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
23982024-05-10T21:34:39.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
23992024-05-10T21:34:39.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
24002024-05-10T21:34:39.027ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
24012024-05-10T21:34:39.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
24022024-05-10T21:34:39.028ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24032024-05-10T21:34:39.031ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
24042024-05-10T21:34:39.048ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
24052024-05-10T21:34:39.048ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24062024-05-10T21:34:39.049ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a75b6f4-6419-42df-900c-815ab1fd5efb", "content-length": "26", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24072024-05-10T21:34:39.049ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24082024-05-10T21:34:39.051ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24092024-05-10T21:34:39.063ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
24102024-05-10T21:34:39.073ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
24112024-05-10T21:34:39.073ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24122024-05-10T21:34:39.074ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cffa328a-d260-4e25-a9b1-4d7eb51c4b60", "content-length": "26", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24132024-05-10T21:34:39.074ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24142024-05-10T21:34:39.075ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24152024-05-10T21:34:39.205ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
24162024-05-10T21:34:39.233ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
24172024-05-10T21:34:39.234ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "764c5810-0fdf-4627-9ec4-1dbf4b21290d", "content-length": "2", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24182024-05-10T21:34:39.235ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24192024-05-10T21:34:39.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24202024-05-10T21:34:39.237ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 377.073104ms iteration = 22
24212024-05-10T21:34:39.241ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
24222024-05-10T21:34:39.241ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24232024-05-10T21:34:39.241ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a168a290-9f52-460c-b961-48e1a4b06b32", "content-length": "26", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24242024-05-10T21:34:39.241ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24252024-05-10T21:34:39.242ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
24262024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
24272024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
24282024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
24292024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/ignition", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dfc16d56-58f8-40a1-b72d-d51b8460f39e", "content-length": "476", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24302024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
24312024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "44576439-0599-4ce6-abb5-3b0045beb411", "content-length": "387", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24322024-05-10T21:34:39.249ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
24332024-05-10T21:34:39.250ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/sp/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "101c359b-83f1-4bc9-bb09-61a41ac23bec", "content-length": "86", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24342024-05-10T21:34:39.250ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
24352024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/sp/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6318404b-aee2-450b-bc3c-7b2e02a7b587", "content-length": "86", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24362024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
24372024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "155b8ad2-3646-4de3-b361-765131299aa5", "content-length": "80", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24382024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
24392024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9ec84f47-4a72-41bf-8b73-cba7f5e064b8", "content-length": "80", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24402024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
24412024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cmpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6e649809-2551-4472-8d1e-405628ba9d20", "content-length": "702", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24422024-05-10T21:34:39.251ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
24432024-05-10T21:34:39.252ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c1f3d62b-7f12-49eb-abc4-e746a13a25a6", "content-length": "727", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
24442024-05-10T21:34:39.252ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
24452024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cba808ff-b688-4f07-bd9b-f9c048cbd76d", "content-length": "729", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24462024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
24472024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/switch/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c95587cd-94bd-46ad-a298-fe3391656dbf", "content-length": "728", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24482024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
24492024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e21bcaba-246b-4213-a63b-05cdfe906bc4", "content-length": "375", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24502024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
24512024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/sp/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "30426522-83d3-493d-85b6-cad8a5fc9e09", "content-length": "84", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24522024-05-10T21:34:39.253ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
24532024-05-10T21:34:39.254ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/sp/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8421cb95-928a-4db5-a584-cec33df5ee66", "content-length": "84", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24542024-05-10T21:34:39.254ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
24552024-05-10T21:34:39.254ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/caboose", query: Some("firmware_slot=0"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "be8ceee9-15a7-406c-8702-515c092f225c", "content-length": "79", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24562024-05-10T21:34:39.254ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
24572024-05-10T21:34:39.254ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/caboose", query: Some("firmware_slot=1"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e85fb2-de14-4a17-9c2d-08b0134ed545", "content-length": "79", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24582024-05-10T21:34:39.254ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
24592024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cmpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1436e829-aef3-4f0e-bf6e-773e42260eeb", "content-length": "702", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24602024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
24612024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "20cbd74a-e7e0-4857-8496-cfda57ff6a5b", "content-length": "727", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24622024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
24632024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3c4520d0-5fd7-46dd-8c70-6a1d754b9ae3", "content-length": "729", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24642024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
24652024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/sp/sled/0/component/rot/cfpa", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "79a598ab-62a9-440f-bc9d-b6cb83241a81", "content-length": "728", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24662024-05-10T21:34:39.255ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24672024-05-10T21:34:39.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24682024-05-10T21:34:39.257ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24692024-05-10T21:34:39.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24702024-05-10T21:34:39.259ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24712024-05-10T21:34:39.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24722024-05-10T21:34:39.261ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24732024-05-10T21:34:39.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
24742024-05-10T21:34:39.263ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
24752024-05-10T21:34:39.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
24762024-05-10T21:34:39.285ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 424.837336ms iteration = 22
24772024-05-10T21:34:39.368ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
24782024-05-10T21:34:39.368ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ee1c5e2f-f4c8-42ab-8e1e-41682e6a7f9f", "date": "Fri, 10 May 2024 21:34:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24792024-05-10T21:34:39.368ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24802024-05-10T21:34:39.370ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2cb602fa-8ed3-493f-be6c-d51690ea8511", "date": "Fri, 10 May 2024 21:34:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24812024-05-10T21:34:39.370ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24822024-05-10T21:34:39.371ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "98afe33d-3be6-42ba-be3f-4740038952d2", "date": "Fri, 10 May 2024 21:34:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24832024-05-10T21:34:39.371ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24842024-05-10T21:34:39.373ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353] }
24852024-05-10T21:34:39.373ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 512.977144ms iteration = 14
24862024-05-10T21:34:39.416ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 3ae06dcf-597c-4d18-b2d7-37c1f02096d7 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
24872024-05-10T21:34:39.420ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
24882024-05-10T21:34:39.420ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7430ad61-c3d8-4876-a54f-7abe21df19bc", "date": "Fri, 10 May 2024 21:34:38 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24892024-05-10T21:34:39.420ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24902024-05-10T21:34:39.422ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b911ed71-7311-4749-afef-8ffca317d386", "date": "Fri, 10 May 2024 21:34:39 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24912024-05-10T21:34:39.422ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24922024-05-10T21:34:39.424ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
24932024-05-10T21:34:39.424ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 564.601696ms iteration = 14
24942024-05-10T21:34:39.456ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
24952024-05-10T21:34:39.456ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
24962024-05-10T21:34:39.482ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 3ae06dcf-597c-4d18-b2d7-37c1f02096d7 background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b2d12891-cafe-4182-9b1d-f069e236c1ac", "date": "Fri, 10 May 2024 21:34:38 GMT"} })
24972024-05-10T21:34:39.484ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:486
24982024-05-10T21:34:39.484ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 623.197844ms
24992024-05-10T21:34:39.484ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 623.204814ms iteration = 4
25002024-05-10T21:34:39.491ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25012024-05-10T21:34:39.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25022024-05-10T21:34:39.492ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
25032024-05-10T21:34:39.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25042024-05-10T21:34:39.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25052024-05-10T21:34:39.499ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
25062024-05-10T21:34:39.499ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "91d6493b-adff-4840-b1fc-e323f56d79b3", "content-length": "121", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25072024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
25082024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
25092024-05-10T21:34:39.499ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
25102024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
25112024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
25122024-05-10T21:34:39.499ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
25132024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
25142024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
25152024-05-10T21:34:39.499ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
25162024-05-10T21:34:39.499ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25172024-05-10T21:34:39.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25182024-05-10T21:34:39.501ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
25192024-05-10T21:34:39.501ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dac264bb-2048-4723-a924-8224479fd042", "content-length": "2", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25202024-05-10T21:34:39.501ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
25212024-05-10T21:34:39.502ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3d632709-38bb-4274-8eb0-e8f5350589e5", "content-length": "51", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25222024-05-10T21:34:39.502ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
25232024-05-10T21:34:39.502ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
25242024-05-10T21:34:39.586ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5f965973-9871-4223-a192-09afda98c005", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25252024-05-10T21:34:39.586ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
25262024-05-10T21:34:39.586ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
25272024-05-10T21:34:39.586ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "187e1fd1-e135-41ec-93a5-9c500c6a1137", "content-length": "467", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25282024-05-10T21:34:39.586ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25292024-05-10T21:34:39.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25302024-05-10T21:34:39.587ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/inventory", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "00e27482-502d-404f-841a-57d83ced98b6", "content-length": "2360", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25312024-05-10T21:34:39.589ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25322024-05-10T21:34:39.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25332024-05-10T21:34:39.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25342024-05-10T21:34:39.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25352024-05-10T21:34:39.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25362024-05-10T21:34:39.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25372024-05-10T21:34:39.594ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/omicron-zones
25382024-05-10T21:34:39.594ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/omicron-zones", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cf22da22-02b1-4a0a-8a3f-8a1ab27d69ee", "content-length": "8504", "date": "Fri, 10 May 2024 21:34:39 GMT"} })
25392024-05-10T21:34:39.600ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 740.014593ms iteration = 22
25402024-05-10T21:34:39.606ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
25412024-05-10T21:34:39.606ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
25422024-05-10T21:34:39.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
25432024-05-10T21:34:39.608ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25442024-05-10T21:34:39.609ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
25452024-05-10T21:34:39.611ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
25462024-05-10T21:34:39.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
25472024-05-10T21:34:39.765ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 40f0f5c8-79d4-45c2-a379-389d9d9e7c34 file = nexus/db-queries/src/db/datastore/inventory.rs:859
25482024-05-10T21:34:39.766ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 40f0f5c8-79d4-45c2-a379-389d9d9e7c34 time_started = 2024-05-10 21:34:39.248459 UTC
25492024-05-10T21:34:39.766ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 906.795449ms iteration = 3
25502024-05-10T21:34:39.766ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Dependency
25512024-05-10T21:34:39.767ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25522024-05-10T21:34:39.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25532024-05-10T21:34:39.768ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25542024-05-10T21:34:39.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25552024-05-10T21:34:39.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25562024-05-10T21:34:39.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25572024-05-10T21:34:39.772ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
25582024-05-10T21:34:39.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25592024-05-10T21:34:39.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25602024-05-10T21:34:39.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25612024-05-10T21:34:39.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25622024-05-10T21:34:39.775ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25632024-05-10T21:34:39.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25642024-05-10T21:34:39.779ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 124 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 4d285268-704e-401c-94fd-7c7c6272fddd response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
25652024-05-10T21:34:39.784ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
25662024-05-10T21:34:39.784ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 17.123319ms iteration = 25
25672024-05-10T21:34:39.894ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::d]:34218
25682024-05-10T21:34:39.894ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34218 req_id = 328256a3-a939-4e73-b8e1-13da6a8d1d4a roles = RoleSet { roles: {} } uri = /metrics/collectors/95738557-5fbe-4760-a2f5-e295833eb800/producers?sort_by=id_ascending
25692024-05-10T21:34:39.895ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34218 req_id = 328256a3-a939-4e73-b8e1-13da6a8d1d4a resource = Database result = Ok(()) uri = /metrics/collectors/95738557-5fbe-4760-a2f5-e295833eb800/producers?sort_by=id_ascending
25702024-05-10T21:34:39.897ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 2564 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34218 req_id = 328256a3-a939-4e73-b8e1-13da6a8d1d4a response_code = 200 uri = /metrics/collectors/95738557-5fbe-4760-a2f5-e295833eb800/producers?sort_by=id_ascending
25712024-05-10T21:34:39.911ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34218 req_id = 09c67fab-0eea-49f0-8343-3d8c313e1fd9 roles = RoleSet { roles: {} } uri = /metrics/collectors/95738557-5fbe-4760-a2f5-e295833eb800/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZDQ2ZmNiMS1jMDdiLTQ4ODAtOTExMi0zODk1ZjJkZjYxZWQifX0%3D
25722024-05-10T21:34:39.911ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34218 req_id = 09c67fab-0eea-49f0-8343-3d8c313e1fd9 resource = Database result = Ok(()) uri = /metrics/collectors/95738557-5fbe-4760-a2f5-e295833eb800/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZDQ2ZmNiMS1jMDdiLTQ4ODAtOTExMi0zODk1ZjJkZjYxZWQifX0%3D
25732024-05-10T21:34:39.913ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 4456 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:34218 req_id = 09c67fab-0eea-49f0-8343-3d8c313e1fd9 response_code = 200 uri = /metrics/collectors/95738557-5fbe-4760-a2f5-e295833eb800/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkZDQ2ZmNiMS1jMDdiLTQ4ODAtOTExMi0zODk1ZjJkZjYxZWQifX0%3D
25742024-05-10T21:34:48.859ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
25752024-05-10T21:34:48.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25762024-05-10T21:34:48.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25772024-05-10T21:34:48.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25782024-05-10T21:34:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25792024-05-10T21:34:48.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25802024-05-10T21:34:48.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25812024-05-10T21:34:48.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25822024-05-10T21:34:48.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25832024-05-10T21:34:48.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25842024-05-10T21:34:48.869ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
25852024-05-10T21:34:48.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25862024-05-10T21:34:48.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25872024-05-10T21:34:48.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
25882024-05-10T21:34:48.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.602787ms iteration = 63
25892024-05-10T21:34:49.916ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 141 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = 895e4f65-8640-4b24-828f-95a621dfd794 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
25902024-05-10T21:34:58.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
25912024-05-10T21:34:58.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25922024-05-10T21:34:58.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25932024-05-10T21:34:58.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25942024-05-10T21:34:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25952024-05-10T21:34:58.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25962024-05-10T21:34:58.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25972024-05-10T21:34:58.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25982024-05-10T21:34:58.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25992024-05-10T21:34:58.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26002024-05-10T21:34:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
26012024-05-10T21:34:58.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26022024-05-10T21:34:58.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26032024-05-10T21:34:58.875ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
26042024-05-10T21:34:58.875ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.294831ms iteration = 64
26052024-05-10T21:34:59.916ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 153 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = defdabf4-c610-405a-a172-bdb594e0a153 response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed
26062024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
26072024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
26082024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26092024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26102024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
26112024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26122024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
26132024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
26142024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26152024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
26162024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26172024-05-10T21:35:08.860ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
26182024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
26192024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
26202024-05-10T21:35:08.860ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
26212024-05-10T21:35:08.860ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
26222024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26232024-05-10T21:35:08.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.57\u{b5}s iteration = 23
26242024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26252024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26262024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
26272024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26282024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26292024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26302024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
26312024-05-10T21:35:08.861ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26322024-05-10T21:35:08.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26332024-05-10T21:35:08.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26342024-05-10T21:35:08.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26352024-05-10T21:35:08.863ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26362024-05-10T21:35:08.863ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1999, tv_nsec: 640986892 } })
26372024-05-10T21:35:08.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26382024-05-10T21:35:08.863ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
26392024-05-10T21:35:08.863ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.845189ms iteration = 23
26402024-05-10T21:35:08.864ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26412024-05-10T21:35:08.864ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:993
26422024-05-10T21:35:08.864ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1999, tv_nsec: 641889121 } })
26432024-05-10T21:35:08.865ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:993
26442024-05-10T21:35:08.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26452024-05-10T21:35:08.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26462024-05-10T21:35:08.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26472024-05-10T21:35:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26482024-05-10T21:35:08.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26492024-05-10T21:35:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26502024-05-10T21:35:08.867ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26512024-05-10T21:35:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26522024-05-10T21:35:08.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26532024-05-10T21:35:08.868ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-3ae06dcf-597c-4d18-b2d7-37c1f02096d7.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1999, tv_nsec: 645768232 } })
26542024-05-10T21:35:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26552024-05-10T21:35:08.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26562024-05-10T21:35:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26572024-05-10T21:35:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26582024-05-10T21:35:08.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26592024-05-10T21:35:08.868ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:993 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
26602024-05-10T21:35:08.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26612024-05-10T21:35:08.869ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
26622024-05-10T21:35:08.869ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26632024-05-10T21:35:08.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26642024-05-10T21:35:08.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26652024-05-10T21:35:08.870ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26662024-05-10T21:35:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26672024-05-10T21:35:08.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26682024-05-10T21:35:08.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26692024-05-10T21:35:08.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26702024-05-10T21:35:08.871ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:469 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
26712024-05-10T21:35:08.871ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 10.05866ms iteration = 23
26722024-05-10T21:35:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26732024-05-10T21:35:08.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26742024-05-10T21:35:08.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26752024-05-10T21:35:08.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 6cd98671-8b12-4197-ac01-c6571c567391, lookup_type: ById(6cd98671-8b12-4197-ac01-c6571c567391) } result = Ok(())
26762024-05-10T21:35:08.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26772024-05-10T21:35:08.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26782024-05-10T21:35:08.879ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 6cd98671-8b12-4197-ac01-c6571c567391 original_time_created = 2024-05-10 21:24:25.420513 UTC target_id = 6cd98671-8b12-4197-ac01-c6571c567391 time_created = 2024-05-10 21:24:25.420513 UTC
26792024-05-10T21:35:08.879ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.362096ms iteration = 65
26802024-05-10T21:35:08.880ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
26812024-05-10T21:35:08.880ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.087821ms iteration = 26
26822024-05-10T21:35:08.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26832024-05-10T21:35:08.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26842024-05-10T21:35:08.884ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26852024-05-10T21:35:08.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26862024-05-10T21:35:08.885ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26872024-05-10T21:35:08.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 3ae06dcf-597c-4d18-b2d7-37c1f02096d7, lookup_type: ById(3ae06dcf-597c-4d18-b2d7-37c1f02096d7) } result = Ok(())
26882024-05-10T21:35:08.888ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26892024-05-10T21:35:08.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26902024-05-10T21:35:08.889ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.035238ms iteration = 23
26912024-05-10T21:35:08.953ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
26922024-05-10T21:35:08.953ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26932024-05-10T21:35:08.953ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1001 zone_address = fd00:1122:3344:101::2
26942024-05-10T21:35:08.953ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26952024-05-10T21:35:08.954ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f1509384-00ca-442d-8c9f-709a6f3df7c6", "content-length": "26", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
26962024-05-10T21:35:08.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26972024-05-10T21:35:08.954ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2e97fc13-a49d-4a3c-acc1-6f61f0d723b5", "content-length": "26", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
26982024-05-10T21:35:08.954ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1006 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26992024-05-10T21:35:08.955ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27002024-05-10T21:35:08.956ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27012024-05-10T21:35:08.957ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1001 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a zone_address = fd00:1122:3344:101::2
27022024-05-10T21:35:08.957ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27032024-05-10T21:35:08.957ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ac08fb75-c3b2-48f4-afe4-3b371b795d64", "content-length": "26", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27042024-05-10T21:35:08.957ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1006 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27052024-05-10T21:35:08.958ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1038 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
27062024-05-10T21:35:09.032ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
27072024-05-10T21:35:09.032ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a7a15131-e1b2-40ea-be7d-47cf7650b46c", "content-length": "2", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27082024-05-10T21:35:09.035ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27092024-05-10T21:35:09.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27102024-05-10T21:35:09.036ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 176.302485ms iteration = 23
27112024-05-10T21:35:09.037ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 176.97795ms iteration = 23
27122024-05-10T21:35:09.094ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27132024-05-10T21:35:09.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27142024-05-10T21:35:09.096ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:144 port = SwitchPort { id: e8fa6949-d2eb-44fd-9146-6855eb0be7ff, rack_id: 58769e48-4e3f-40ad-a95a-fd916e51fe8a, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(1261acce-db1c-45d0-b771-991a8712f536) } rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
27152024-05-10T21:35:09.096ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27162024-05-10T21:35:09.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27172024-05-10T21:35:09.102ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
27182024-05-10T21:35:09.103ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5a87b4dc-1cdc-4f07-bc76-4e369144d026", "content-length": "121", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27192024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1643 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
27202024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1670 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 1261acce-db1c-45d0-b771-991a8712f536
27212024-05-10T21:35:09.103ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1782 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
27222024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:369 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
27232024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:373 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
27242024-05-10T21:35:09.103ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1511 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
27252024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:382 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
27262024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:388 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a routes = {}
27272024-05-10T21:35:09.103ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:408 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a
27282024-05-10T21:35:09.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27292024-05-10T21:35:09.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27302024-05-10T21:35:09.105ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
27312024-05-10T21:35:09.105ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4c84be75-68f7-4ac4-8945-54b3d8ce7695", "content-length": "2", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27322024-05-10T21:35:09.105ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
27332024-05-10T21:35:09.105ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "23f0f476-5d83-48a2-b779-c7bde42055bc", "content-length": "51", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27342024-05-10T21:35:09.105ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.2.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:457 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch_location = Switch0
27352024-05-10T21:35:09.105ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27362024-05-10T21:35:09.153ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "03413f0a-bc18-4d7a-8cc9-35461eed8b46", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27372024-05-10T21:35:09.153ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:759 rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a switch = Switch0
27382024-05-10T21:35:09.153ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27392024-05-10T21:35:09.156ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 58769e48-4e3f-40ad-a95a-fd916e51fe8a result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4d31bfcf-8420-4ca9-82f8-61a2f89e2b64", "content-length": "467", "date": "Fri, 10 May 2024 21:35:08 GMT"} })
27402024-05-10T21:35:09.156ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27412024-05-10T21:35:09.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27422024-05-10T21:35:09.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27432024-05-10T21:35:09.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27442024-05-10T21:35:09.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27452024-05-10T21:35:09.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27462024-05-10T21:35:09.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27472024-05-10T21:35:09.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27482024-05-10T21:35:09.170ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 309.658161ms iteration = 23
27492024-05-10T21:35:09.917ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 720 local_addr = [fd00:1122:3344:101::c]:52586 method = GET remote_addr = [fd00:1122:3344:101::d]:61594 req_id = d711ecac-acb2-48c3-8422-1f84b5b6922b response_code = 200 uri = //dd46fcb1-c07b-4880-9112-3895f2df61ed