Max level shown:
12024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
22024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
42024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
52024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
62024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
72024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
92024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
102024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
122024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
132024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
142024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
152024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
162024-05-30T04:30:02.136ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:20:02.136613529 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
172024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
182024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
192024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
202024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
212024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
222024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
232024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
242024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
252024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
262024-05-30T04:30:02.136ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
272024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
282024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
292024-05-30T04:30:02.136ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
302024-05-30T04:30:02.136ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
312024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
322024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.04\u{b5}s iteration = 5
332024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
342024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
352024-05-30T04:30:02.136ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
362024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.65\u{b5}s iteration = 4
372024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
382024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
392024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
402024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
412024-05-30T04:30:02.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
422024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
432024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
442024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
452024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
462024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
472024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
482024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
492024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
502024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
512024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
522024-05-30T04:30:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
532024-05-30T04:30:02.138ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 253580173 } })
542024-05-30T04:30:02.138ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 253636953 } })
552024-05-30T04:30:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
562024-05-30T04:30:02.139ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
572024-05-30T04:30:02.139ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.837603ms iteration = 5
582024-05-30T04:30:02.139ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
592024-05-30T04:30:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
602024-05-30T04:30:02.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
612024-05-30T04:30:02.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
622024-05-30T04:30:02.141ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
632024-05-30T04:30:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.553223ms iteration = 4
642024-05-30T04:30:02.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
652024-05-30T04:30:02.141ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:20:02.136613529 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
662024-05-30T04:30:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.685085ms iteration = 3
672024-05-30T04:30:02.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
682024-05-30T04:30:02.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
692024-05-30T04:30:02.142ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
702024-05-30T04:30:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.700116ms iteration = 4
712024-05-30T04:30:02.142ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 257629969 } })
722024-05-30T04:30:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
732024-05-30T04:30:02.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
742024-05-30T04:30:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
752024-05-30T04:30:02.143ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
762024-05-30T04:30:02.143ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
772024-05-30T04:30:02.143ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
782024-05-30T04:30:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
792024-05-30T04:30:02.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
802024-05-30T04:30:02.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 259279688 } })
812024-05-30T04:30:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
822024-05-30T04:30:02.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
832024-05-30T04:30:02.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
842024-05-30T04:30:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
852024-05-30T04:30:02.145ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
862024-05-30T04:30:02.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
872024-05-30T04:30:02.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
882024-05-30T04:30:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
892024-05-30T04:30:02.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
902024-05-30T04:30:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
912024-05-30T04:30:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
922024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
932024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
942024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
952024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
962024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
972024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
982024-05-30T04:30:02.146ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 261930709 } })
992024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1002024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1012024-05-30T04:30:02.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1022024-05-30T04:30:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1032024-05-30T04:30:02.147ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1042024-05-30T04:30:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1052024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1062024-05-30T04:30:02.148ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
1072024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1082024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1092024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1102024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1112024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1122024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1132024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1142024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1152024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1162024-05-30T04:30:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1172024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1182024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1192024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1202024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1212024-05-30T04:30:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1222024-05-30T04:30:02.150ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1232024-05-30T04:30:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1242024-05-30T04:30:02.152ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1252024-05-30T04:30:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1262024-05-30T04:30:02.153ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1272024-05-30T04:30:02.153ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1282024-05-30T04:30:02.153ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1292024-05-30T04:30:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1302024-05-30T04:30:02.153ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
1312024-05-30T04:30:02.153ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 17.232849ms iteration = 5
1322024-05-30T04:30:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1332024-05-30T04:30:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1342024-05-30T04:30:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1352024-05-30T04:30:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1362024-05-30T04:30:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1372024-05-30T04:30:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
1382024-05-30T04:30:02.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1392024-05-30T04:30:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1402024-05-30T04:30:02.156ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1412024-05-30T04:30:02.157ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
1422024-05-30T04:30:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1432024-05-30T04:30:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1442024-05-30T04:30:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1452024-05-30T04:30:02.157ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1462024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1472024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1482024-05-30T04:30:02.158ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
1492024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.635739ms iteration = 6
1502024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1512024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1522024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1542024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1552024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1562024-05-30T04:30:02.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1572024-05-30T04:30:02.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1582024-05-30T04:30:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1592024-05-30T04:30:02.164ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1602024-05-30T04:30:02.164ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1612024-05-30T04:30:02.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1622024-05-30T04:30:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1632024-05-30T04:30:02.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1642024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1652024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1662024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1672024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1682024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
1692024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.29071ms iteration = 13
1702024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1712024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1722024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1732024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1742024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1752024-05-30T04:30:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1762024-05-30T04:30:02.184ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1772024-05-30T04:30:02.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1782024-05-30T04:30:02.186ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1792024-05-30T04:30:02.186ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
1802024-05-30T04:30:02.186ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1812024-05-30T04:30:02.186ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
1822024-05-30T04:30:02.186ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 50.143937ms iteration = 4
1832024-05-30T04:30:02.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1842024-05-30T04:30:02.188ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1852024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
1862024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1872024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1882024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
1892024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1902024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
1912024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 56.839614ms iteration = 4
1922024-05-30T04:30:02.196ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 59.524045ms iteration = 5
1932024-05-30T04:30:02.200ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
1942024-05-30T04:30:02.200ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
1952024-05-30T04:30:02.200ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.409609ms iteration = 3
1962024-05-30T04:30:02.231ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1972024-05-30T04:30:02.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1982024-05-30T04:30:02.233ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1992024-05-30T04:30:02.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2002024-05-30T04:30:02.236ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2012024-05-30T04:30:02.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2022024-05-30T04:30:02.246ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
2032024-05-30T04:30:02.246ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 109.53626ms iteration = 4
2042024-05-30T04:30:02.306ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
2052024-05-30T04:30:02.306ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2062024-05-30T04:30:02.306ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3ea049eb-156d-4c54-9e0b-afc547eca27f", "content-length": "26", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2072024-05-30T04:30:02.306ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2082024-05-30T04:30:02.308ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2092024-05-30T04:30:02.310ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2102024-05-30T04:30:02.329ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
2112024-05-30T04:30:02.329ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2122024-05-30T04:30:02.329ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "001fc289-c4b5-4d6a-9f30-029bb7205014", "content-length": "26", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2132024-05-30T04:30:02.329ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2142024-05-30T04:30:02.331ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
2152024-05-30T04:30:02.421ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2162024-05-30T04:30:02.422ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8a34612b-6498-49d9-8fd0-5129f4800f7e", "content-length": "691", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2172024-05-30T04:30:02.422ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
2182024-05-30T04:30:02.422ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
2192024-05-30T04:30:02.422ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
2202024-05-30T04:30:02.423ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 287.267685ms iteration = 5
2212024-05-30T04:30:02.442ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
2222024-05-30T04:30:02.442ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2232024-05-30T04:30:02.442ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d0c93377-d851-48eb-8d73-eb09a41ec5de", "content-length": "26", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2242024-05-30T04:30:02.443ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2252024-05-30T04:30:02.444ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2262024-05-30T04:30:02.446ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
2272024-05-30T04:30:02.475ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 339.160492ms iteration = 5
2282024-05-30T04:30:02.477ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2292024-05-30T04:30:02.477ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a10ef055-24dd-4595-8bd2-42694c689a82", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2302024-05-30T04:30:02.477ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2312024-05-30T04:30:02.479ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b26fcffe-9b2d-4802-925a-4cc9c6561e9f", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2322024-05-30T04:30:02.479ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2332024-05-30T04:30:02.481ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2342024-05-30T04:30:02.481ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 345.314253ms iteration = 5
2352024-05-30T04:30:02.564ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2362024-05-30T04:30:02.654ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2372024-05-30T04:30:02.655ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2409b3b9-30b2-4ccb-abbe-05b92ce01e66", "content-length": "1", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2382024-05-30T04:30:02.657ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2392024-05-30T04:30:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2402024-05-30T04:30:02.659ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 523.086039ms iteration = 5
2412024-05-30T04:30:02.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2422024-05-30T04:30:02.667ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "adcb7988-9aac-4918-b95c-8f1169cee989", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2432024-05-30T04:30:02.667ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2442024-05-30T04:30:02.670ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "23bb7644-deb6-4161-9c1a-2c523374df2e", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2452024-05-30T04:30:02.670ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2462024-05-30T04:30:02.671ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bf702494-063f-4b4d-a7fe-1e2bbc869063", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2472024-05-30T04:30:02.671ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2482024-05-30T04:30:02.673ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2492024-05-30T04:30:02.673ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 536.801246ms iteration = 5
2502024-05-30T04:30:02.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2512024-05-30T04:30:02.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2522024-05-30T04:30:02.720ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2532024-05-30T04:30:02.720ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2542024-05-30T04:30:02.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2552024-05-30T04:30:02.728ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2562024-05-30T04:30:02.729ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6738722d-c94d-49a5-9cbd-7627f610e7fc", "content-length": "121", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2572024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
2582024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
2592024-05-30T04:30:02.729ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2602024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
2612024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
2622024-05-30T04:30:02.729ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2632024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
2642024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
2652024-05-30T04:30:02.729ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
2662024-05-30T04:30:02.729ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2672024-05-30T04:30:02.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2682024-05-30T04:30:02.831ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2692024-05-30T04:30:02.831ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6ecbd19b-25d3-4afc-8494-0f4428accb15", "content-length": "2", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2702024-05-30T04:30:02.831ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2712024-05-30T04:30:02.831ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1c5d190f-721d-48dc-9d1d-1df03e24fa2f", "content-length": "52", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2722024-05-30T04:30:02.831ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2732024-05-30T04:30:02.831ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2742024-05-30T04:30:02.965ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "563a0d2e-5780-4f0a-a1d5-0d2918c3f2c4", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2752024-05-30T04:30:02.965ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
2762024-05-30T04:30:02.965ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2772024-05-30T04:30:02.965ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "12a5e092-691d-410e-86b9-f918319fef53", "content-length": "467", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2782024-05-30T04:30:02.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2792024-05-30T04:30:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2802024-05-30T04:30:02.968ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2812024-05-30T04:30:02.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2822024-05-30T04:30:02.971ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2832024-05-30T04:30:02.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2842024-05-30T04:30:02.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2852024-05-30T04:30:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2862024-05-30T04:30:02.979ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 843.05352ms iteration = 5
2872024-05-30T04:30:08.676ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 437 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = f98a8623-75a6-47ac-b7f1-d7af8f0c2bdc response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
2882024-05-30T04:30:12.136ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
2892024-05-30T04:30:12.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2902024-05-30T04:30:12.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2912024-05-30T04:30:12.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2922024-05-30T04:30:12.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2932024-05-30T04:30:12.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2942024-05-30T04:30:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2952024-05-30T04:30:12.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2962024-05-30T04:30:12.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2972024-05-30T04:30:12.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2982024-05-30T04:30:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
2992024-05-30T04:30:12.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3002024-05-30T04:30:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3012024-05-30T04:30:12.154ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
3022024-05-30T04:30:12.154ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.278001ms iteration = 14
3032024-05-30T04:30:18.676ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 416 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 36795413-0192-4682-a5fb-d7171853a8b6 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
3042024-05-30T04:30:22.136ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
3052024-05-30T04:30:22.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3062024-05-30T04:30:22.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3072024-05-30T04:30:22.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3082024-05-30T04:30:22.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3092024-05-30T04:30:22.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3102024-05-30T04:30:22.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3112024-05-30T04:30:22.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3122024-05-30T04:30:22.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3132024-05-30T04:30:22.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3142024-05-30T04:30:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
3152024-05-30T04:30:22.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3162024-05-30T04:30:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3172024-05-30T04:30:22.152ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
3182024-05-30T04:30:22.152ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.222066ms iteration = 15
3192024-05-30T04:30:28.675ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 408 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 4c520195-d54b-4a9f-95f3-42300e9e6fb8 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
3202024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
3212024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
3222024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3232024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
3242024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3252024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3262024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
3272024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
3282024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3292024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3302024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
3312024-05-30T04:30:32.137ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
3322024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
3332024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
3342024-05-30T04:30:32.137ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
3352024-05-30T04:30:32.137ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
3362024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.99\u{b5}s iteration = 6
3372024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3382024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
3392024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
3402024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3412024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3422024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3432024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3442024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3452024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3462024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3472024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3482024-05-30T04:30:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3492024-05-30T04:30:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3502024-05-30T04:30:32.139ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3512024-05-30T04:30:32.139ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.262396ms iteration = 6
3522024-05-30T04:30:32.139ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3532024-05-30T04:30:32.139ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3542024-05-30T04:30:32.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3552024-05-30T04:30:32.140ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1054, tv_nsec: 255393583 } })
3562024-05-30T04:30:32.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572024-05-30T04:30:32.140ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1054, tv_nsec: 255645546 } })
3582024-05-30T04:30:32.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3592024-05-30T04:30:32.141ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
3602024-05-30T04:30:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3612024-05-30T04:30:32.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3622024-05-30T04:30:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3632024-05-30T04:30:32.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3642024-05-30T04:30:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3652024-05-30T04:30:32.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
3662024-05-30T04:30:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3672024-05-30T04:30:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3682024-05-30T04:30:32.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3692024-05-30T04:30:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3702024-05-30T04:30:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3712024-05-30T04:30:32.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3722024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3732024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3742024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3752024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3762024-05-30T04:30:32.145ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
3772024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3782024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3792024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3802024-05-30T04:30:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3812024-05-30T04:30:32.146ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1054, tv_nsec: 261198930 } })
3822024-05-30T04:30:32.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3832024-05-30T04:30:32.146ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
3842024-05-30T04:30:32.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3852024-05-30T04:30:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3862024-05-30T04:30:32.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3872024-05-30T04:30:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3882024-05-30T04:30:32.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3892024-05-30T04:30:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3902024-05-30T04:30:32.149ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
3912024-05-30T04:30:32.149ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 11.972478ms iteration = 6
3922024-05-30T04:30:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
3932024-05-30T04:30:32.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3942024-05-30T04:30:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3952024-05-30T04:30:32.154ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
3962024-05-30T04:30:32.154ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 16.962295ms iteration = 7
3972024-05-30T04:30:32.155ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3982024-05-30T04:30:32.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3992024-05-30T04:30:32.156ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4002024-05-30T04:30:32.157ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
4012024-05-30T04:30:32.157ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.055071ms iteration = 16
4022024-05-30T04:30:32.158ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4032024-05-30T04:30:32.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4042024-05-30T04:30:32.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4052024-05-30T04:30:32.159ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4062024-05-30T04:30:32.161ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4072024-05-30T04:30:32.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4082024-05-30T04:30:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4092024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4102024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4112024-05-30T04:30:32.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4122024-05-30T04:30:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4132024-05-30T04:30:32.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4142024-05-30T04:30:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
4152024-05-30T04:30:32.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4162024-05-30T04:30:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4172024-05-30T04:30:32.171ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.927151ms iteration = 6
4182024-05-30T04:30:32.236ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
4192024-05-30T04:30:32.236ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4202024-05-30T04:30:32.236ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "361a4327-be00-44cf-8981-db0714ba207d", "content-length": "26", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4212024-05-30T04:30:32.236ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4222024-05-30T04:30:32.236ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
4232024-05-30T04:30:32.236ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4242024-05-30T04:30:32.236ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "238b3574-5a1b-45e7-85ca-876bd21de17d", "content-length": "26", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4252024-05-30T04:30:32.236ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4262024-05-30T04:30:32.237ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
4272024-05-30T04:30:32.237ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4282024-05-30T04:30:32.238ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4292024-05-30T04:30:32.238ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4302024-05-30T04:30:32.239ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "54c9f828-9e24-4dd3-9db1-d087f8bd17dc", "content-length": "26", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4312024-05-30T04:30:32.239ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4322024-05-30T04:30:32.240ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
4332024-05-30T04:30:32.264ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4342024-05-30T04:30:32.264ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d39e23e8-4211-44eb-a0fb-91be414658af", "content-length": "691", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4352024-05-30T04:30:32.264ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
4362024-05-30T04:30:32.264ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
4372024-05-30T04:30:32.264ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
4382024-05-30T04:30:32.266ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 128.840932ms iteration = 6
4392024-05-30T04:30:32.333ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4402024-05-30T04:30:32.334ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6d4348f4-bb93-4f1d-a4a4-c27710ea4f34", "content-length": "1", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4412024-05-30T04:30:32.334ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 197.028357ms iteration = 6
4422024-05-30T04:30:32.335ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4432024-05-30T04:30:32.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4442024-05-30T04:30:32.337ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 199.831739ms iteration = 6
4452024-05-30T04:30:32.393ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4462024-05-30T04:30:32.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4472024-05-30T04:30:32.395ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4482024-05-30T04:30:32.395ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4492024-05-30T04:30:32.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4502024-05-30T04:30:32.402ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4512024-05-30T04:30:32.403ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "72e9eafa-2bf1-4274-b692-1a02dc451b2d", "content-length": "121", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4522024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4532024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
4542024-05-30T04:30:32.403ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4552024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
4562024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
4572024-05-30T04:30:32.403ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4582024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
4592024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
4602024-05-30T04:30:32.403ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
4612024-05-30T04:30:32.403ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4622024-05-30T04:30:32.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4632024-05-30T04:30:32.405ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4642024-05-30T04:30:32.405ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d59be8ac-7365-40e1-96e3-a7ebc041d93e", "content-length": "2", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4652024-05-30T04:30:32.405ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4662024-05-30T04:30:32.406ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "007062d4-6f53-40db-a7dd-01ba339b7900", "content-length": "52", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4672024-05-30T04:30:32.406ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4682024-05-30T04:30:32.406ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4692024-05-30T04:30:32.456ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6910e2ff-3bc0-4579-a315-44a899d4a915", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4702024-05-30T04:30:32.456ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
4712024-05-30T04:30:32.456ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4722024-05-30T04:30:32.456ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "96e05679-e7fd-46b1-9801-a33036ec9a00", "content-length": "467", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4732024-05-30T04:30:32.456ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4742024-05-30T04:30:32.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4752024-05-30T04:30:32.460ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4762024-05-30T04:30:32.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4772024-05-30T04:30:32.463ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4782024-05-30T04:30:32.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4792024-05-30T04:30:32.464ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4802024-05-30T04:30:32.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4812024-05-30T04:30:32.469ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 332.270393ms iteration = 6
4822024-05-30T04:30:35.440ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:63773
4832024-05-30T04:30:35.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:63773 req_id = 6b2e7377-27ac-46c7-b8c4-2f7f64428e0a roles = RoleSet { roles: {} } uri = /metrics/producers
4842024-05-30T04:30:35.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:63773 req_id = 6b2e7377-27ac-46c7-b8c4-2f7f64428e0a resource = Database result = Ok(()) uri = /metrics/producers
4852024-05-30T04:30:35.471ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
4862024-05-30T04:30:35.471ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:63773 req_id = 6b2e7377-27ac-46c7-b8c4-2f7f64428e0a roles = RoleSet { roles: {} } uri = /metrics/producers
4872024-05-30T04:30:35.472ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:63773 req_id = 6b2e7377-27ac-46c7-b8c4-2f7f64428e0a resource = Database result = Ok(()) uri = /metrics/producers
4882024-05-30T04:30:35.476ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
4892024-05-30T04:30:35.476ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "634f7f8d-83a4-403d-af8f-89f6add7df5a", "date": "Thu, 30 May 2024 04:30:35 GMT"} })
4902024-05-30T04:30:35.476ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 432db3ab-1c9c-4d65-aed9-91b1c693bf73
4912024-05-30T04:30:35.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 37304 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:63773 req_id = 6b2e7377-27ac-46c7-b8c4-2f7f64428e0a response_code = 201 uri = /metrics/producers
4922024-05-30T04:30:36.613ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:59427
4932024-05-30T04:30:36.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:59427 req_id = 121535a8-2ce1-41b3-9a2e-67b0ce224627 roles = RoleSet { roles: {} } uri = /metrics/producers
4942024-05-30T04:30:36.614ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:59427 req_id = 121535a8-2ce1-41b3-9a2e-67b0ce224627 resource = Database result = Ok(()) uri = /metrics/producers
4952024-05-30T04:30:36.647ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
4962024-05-30T04:30:36.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:59427 req_id = 121535a8-2ce1-41b3-9a2e-67b0ce224627 roles = RoleSet { roles: {} } uri = /metrics/producers
4972024-05-30T04:30:36.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:59427 req_id = 121535a8-2ce1-41b3-9a2e-67b0ce224627 resource = Database result = Ok(()) uri = /metrics/producers
4982024-05-30T04:30:36.655ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
4992024-05-30T04:30:36.656ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "506bba70-ab41-4b56-a7aa-a3521614617c", "date": "Thu, 30 May 2024 04:30:36 GMT"} })
5002024-05-30T04:30:36.656ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 73ed95a1-f279-488a-9386-6d3e4371db4d
5012024-05-30T04:30:36.658ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 44873 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:59427 req_id = 121535a8-2ce1-41b3-9a2e-67b0ce224627 response_code = 201 uri = /metrics/producers
5022024-05-30T04:30:38.676ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 452 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 6acca645-db7a-45bb-aee0-0141ef626bfe response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
5032024-05-30T04:30:42.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
5042024-05-30T04:30:42.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5052024-05-30T04:30:42.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5062024-05-30T04:30:42.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5072024-05-30T04:30:42.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5082024-05-30T04:30:42.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5092024-05-30T04:30:42.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5102024-05-30T04:30:42.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5112024-05-30T04:30:42.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5122024-05-30T04:30:42.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5132024-05-30T04:30:42.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
5142024-05-30T04:30:42.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5152024-05-30T04:30:42.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5162024-05-30T04:30:42.155ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
5172024-05-30T04:30:42.155ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.011827ms iteration = 17
5182024-05-30T04:30:48.676ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 434 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 35ccc169-d6ed-4547-a913-0de36efede63 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
5192024-05-30T04:30:52.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
5202024-05-30T04:30:52.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5212024-05-30T04:30:52.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5222024-05-30T04:30:52.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5232024-05-30T04:30:52.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5242024-05-30T04:30:52.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5252024-05-30T04:30:52.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5262024-05-30T04:30:52.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5272024-05-30T04:30:52.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5282024-05-30T04:30:52.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5292024-05-30T04:30:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
5302024-05-30T04:30:52.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5312024-05-30T04:30:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5322024-05-30T04:30:52.154ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
5332024-05-30T04:30:52.154ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.229927ms iteration = 18
5342024-05-30T04:30:58.647ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
5352024-05-30T04:30:58.647ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
5362024-05-30T04:30:58.676ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
5372024-05-30T04:30:58.676ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:47431
5382024-05-30T04:30:58.677ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 427 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 068e73d6-3039-4aad-a1c0-b6fcb17e44ca response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
5392024-05-30T04:30:58.677ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47431 req_id = 65681920-1c8e-442a-bf36-f853d400ff1e roles = RoleSet { roles: {} } uri = /metrics/producers
5402024-05-30T04:30:58.678ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47431 req_id = 65681920-1c8e-442a-bf36-f853d400ff1e resource = Database result = Ok(()) uri = /metrics/producers
5412024-05-30T04:30:58.710ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
5422024-05-30T04:30:58.710ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47431 req_id = 65681920-1c8e-442a-bf36-f853d400ff1e roles = RoleSet { roles: {} } uri = /metrics/producers
5432024-05-30T04:30:58.710ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47431 req_id = 65681920-1c8e-442a-bf36-f853d400ff1e resource = Database result = Ok(()) uri = /metrics/producers
5442024-05-30T04:30:58.715ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
5452024-05-30T04:30:58.715ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "baa0ff09-0a11-4d82-a92d-49f4a6ac4822", "date": "Thu, 30 May 2024 04:30:58 GMT"} })
5462024-05-30T04:30:58.715ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 31c86263-9349-4603-aaa0-6dc44827df57
5472024-05-30T04:30:58.716ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 39458 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:47431 req_id = 65681920-1c8e-442a-bf36-f853d400ff1e response_code = 201 uri = /metrics/producers
5482024-05-30T04:30:58.716ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "65681920-1c8e-442a-bf36-f853d400ff1e", "content-length": "41", "date": "Thu, 30 May 2024 04:30:58 GMT"} })
5492024-05-30T04:30:58.717ZDEBGnexus (producer-registration-task): registered with nexus successfully
5502024-05-30T04:30:58.717ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
5512024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
5522024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5532024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
5542024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
5552024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
5562024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
5572024-05-30T04:31:02.137ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5582024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5592024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
5602024-05-30T04:31:02.137ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5612024-05-30T04:31:02.137ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5622024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
5632024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.171\u{b5}s iteration = 7
5642024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
5652024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
5662024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5672024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5682024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5692024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
5702024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
5712024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5722024-05-30T04:31:02.137ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
5732024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.05\u{b5}s iteration = 5
5742024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
5752024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
5762024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5772024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5782024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5792024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
5802024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
5812024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
5822024-05-30T04:31:02.137ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:21:02.137629700 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
5832024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5842024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
5852024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5862024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
5872024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
5882024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5892024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5902024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
5912024-05-30T04:31:02.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5922024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5932024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5942024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5952024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5962024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5972024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5982024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5992024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6002024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6012024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6022024-05-30T04:31:02.138ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 252574942 } })
6032024-05-30T04:31:02.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6042024-05-30T04:31:02.138ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 252614612 } })
6052024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6062024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6072024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6082024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6092024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6102024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
6112024-05-30T04:31:02.139ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.347297ms iteration = 5
6122024-05-30T04:31:02.140ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
6132024-05-30T04:31:02.140ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.46756ms iteration = 5
6142024-05-30T04:31:02.141ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:21:02.137629700 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
6152024-05-30T04:31:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.599771ms iteration = 4
6162024-05-30T04:31:02.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6172024-05-30T04:31:02.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6182024-05-30T04:31:02.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6192024-05-30T04:31:02.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6202024-05-30T04:31:02.142ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 256585858 } })
6212024-05-30T04:31:02.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6222024-05-30T04:31:02.142ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6232024-05-30T04:31:02.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6242024-05-30T04:31:02.142ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
6252024-05-30T04:31:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.069788ms iteration = 7
6262024-05-30T04:31:02.142ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
6272024-05-30T04:31:02.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6282024-05-30T04:31:02.144ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6292024-05-30T04:31:02.144ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
6302024-05-30T04:31:02.144ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6312024-05-30T04:31:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6322024-05-30T04:31:02.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
6332024-05-30T04:31:02.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
6342024-05-30T04:31:02.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
6352024-05-30T04:31:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
6362024-05-30T04:31:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6372024-05-30T04:31:02.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6382024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
6392024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6402024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6412024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6422024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6432024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6442024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6452024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6462024-05-30T04:31:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6472024-05-30T04:31:02.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6482024-05-30T04:31:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6492024-05-30T04:31:02.147ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6502024-05-30T04:31:02.147ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6512024-05-30T04:31:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6522024-05-30T04:31:02.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 262247783 } })
6532024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6542024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6552024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6562024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6572024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6582024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6592024-05-30T04:31:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6602024-05-30T04:31:02.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6612024-05-30T04:31:02.149ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
6622024-05-30T04:31:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6632024-05-30T04:31:02.149ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 263949643 } })
6642024-05-30T04:31:02.150ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
6652024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6662024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6672024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6682024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6692024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6702024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6712024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6722024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6732024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6742024-05-30T04:31:02.152ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
6752024-05-30T04:31:02.152ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 15.326597ms iteration = 7
6762024-05-30T04:31:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6772024-05-30T04:31:02.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782024-05-30T04:31:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6792024-05-30T04:31:02.156ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6802024-05-30T04:31:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6812024-05-30T04:31:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
6822024-05-30T04:31:02.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6832024-05-30T04:31:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6842024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6852024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6862024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6872024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6882024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6892024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6902024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6912024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6922024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6942024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6952024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6962024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6972024-05-30T04:31:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6982024-05-30T04:31:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6992024-05-30T04:31:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7002024-05-30T04:31:02.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7012024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7022024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7032024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7042024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7052024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7062024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7072024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
7082024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.75918ms iteration = 19
7092024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
7102024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7112024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7122024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7132024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7142024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
7152024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
7162024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7172024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7182024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7192024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7202024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7212024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
7222024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
7232024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
7242024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 37.917396ms iteration = 5
7252024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7262024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7272024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7282024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7292024-05-30T04:31:02.177ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7302024-05-30T04:31:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
7312024-05-30T04:31:02.180ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7322024-05-30T04:31:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7332024-05-30T04:31:02.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
7342024-05-30T04:31:02.181ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7352024-05-30T04:31:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7362024-05-30T04:31:02.182ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
7372024-05-30T04:31:02.182ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7382024-05-30T04:31:02.182ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
7392024-05-30T04:31:02.182ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 44.702974ms iteration = 5
7402024-05-30T04:31:02.184ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 46.671457ms iteration = 7
7412024-05-30T04:31:02.223ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
7422024-05-30T04:31:02.223ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 86.375903ms iteration = 8
7432024-05-30T04:31:02.226ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7442024-05-30T04:31:02.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7452024-05-30T04:31:02.228ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462024-05-30T04:31:02.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
7472024-05-30T04:31:02.230ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7482024-05-30T04:31:02.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7492024-05-30T04:31:02.242ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
7502024-05-30T04:31:02.242ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 104.441912ms iteration = 5
7512024-05-30T04:31:02.315ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7522024-05-30T04:31:02.316ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
7532024-05-30T04:31:02.316ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7542024-05-30T04:31:02.316ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6c45a727-7331-4313-9039-b5980a9a4798", "content-length": "26", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7552024-05-30T04:31:02.316ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7562024-05-30T04:31:02.317ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7572024-05-30T04:31:02.327ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
7582024-05-30T04:31:02.327ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
7592024-05-30T04:31:02.327ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 182.105325ms iteration = 4
7602024-05-30T04:31:02.415ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7612024-05-30T04:31:02.446ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
7622024-05-30T04:31:02.446ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7632024-05-30T04:31:02.447ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d2e021e2-3950-4ab5-a572-37fd9cfae332", "content-length": "26", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7642024-05-30T04:31:02.447ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7652024-05-30T04:31:02.448ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7662024-05-30T04:31:02.478ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
7672024-05-30T04:31:02.478ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7682024-05-30T04:31:02.479ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "005e5d59-84e0-42fe-8ade-e2b30e08bc5c", "content-length": "26", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7692024-05-30T04:31:02.479ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7702024-05-30T04:31:02.480ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7712024-05-30T04:31:02.480ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
7722024-05-30T04:31:02.480ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c847063b-59f3-4b91-bdb2-16da27ed7236", "content-length": "1", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7732024-05-30T04:31:02.481ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7742024-05-30T04:31:02.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7752024-05-30T04:31:02.483ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 345.794089ms iteration = 7
7762024-05-30T04:31:02.503ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7772024-05-30T04:31:02.529ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7782024-05-30T04:31:02.529ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "45af455b-3aaf-4c6a-987b-446629182427", "content-length": "691", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7792024-05-30T04:31:02.529ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
7802024-05-30T04:31:02.529ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
7812024-05-30T04:31:02.529ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
7822024-05-30T04:31:02.531ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 393.744681ms iteration = 7
7832024-05-30T04:31:02.628ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7842024-05-30T04:31:02.629ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fb1cf4e8-5193-4b54-8d88-d501f0f85b80", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7852024-05-30T04:31:02.629ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7862024-05-30T04:31:02.630ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "edb0ef41-fa7a-46f8-bcb5-cefaae59c1a2", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7872024-05-30T04:31:02.630ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7882024-05-30T04:31:02.632ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6c36f37e-bd3e-484c-8c2b-eefc00fab3cc", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7892024-05-30T04:31:02.632ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7902024-05-30T04:31:02.634ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7912024-05-30T04:31:02.634ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 497.357172ms iteration = 6
7922024-05-30T04:31:02.687ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7932024-05-30T04:31:02.687ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f2dd9921-0e8e-4ec1-bd02-05cce2d565e0", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7942024-05-30T04:31:02.687ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7952024-05-30T04:31:02.689ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cf9830e8-7ae6-4a8f-800a-96180aa2a6e3", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7962024-05-30T04:31:02.689ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7972024-05-30T04:31:02.690ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7982024-05-30T04:31:02.690ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 552.68829ms iteration = 6
7992024-05-30T04:31:02.697ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 559.920863ms iteration = 7
8002024-05-30T04:31:02.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8012024-05-30T04:31:02.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8022024-05-30T04:31:02.735ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
8032024-05-30T04:31:02.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8042024-05-30T04:31:02.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8052024-05-30T04:31:02.743ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8062024-05-30T04:31:02.743ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "180d0c9a-f8d5-42a7-bfbf-15c65f8a3fa6", "content-length": "121", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
8072024-05-30T04:31:02.743ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
8082024-05-30T04:31:02.743ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
8092024-05-30T04:31:02.744ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
8102024-05-30T04:31:02.744ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
8112024-05-30T04:31:02.744ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
8122024-05-30T04:31:02.744ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
8132024-05-30T04:31:02.744ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
8142024-05-30T04:31:02.744ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
8152024-05-30T04:31:02.744ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
8162024-05-30T04:31:02.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8172024-05-30T04:31:02.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8182024-05-30T04:31:02.746ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8192024-05-30T04:31:02.746ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "34b82a3d-4307-42dd-9b25-1b4953677036", "content-length": "2", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
8202024-05-30T04:31:02.746ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8212024-05-30T04:31:02.746ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3327d23a-e145-43d2-babf-f7d230b9549a", "content-length": "52", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
8222024-05-30T04:31:02.746ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
8232024-05-30T04:31:02.746ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8242024-05-30T04:31:02.794ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "92948f06-3d81-40a6-bd05-19dd0dc19720", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
8252024-05-30T04:31:02.794ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
8262024-05-30T04:31:02.794ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8272024-05-30T04:31:02.795ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8d54d9fb-effa-42f1-9445-47bc6d60c1d4", "content-length": "467", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
8282024-05-30T04:31:02.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8292024-05-30T04:31:02.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8302024-05-30T04:31:02.798ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8312024-05-30T04:31:02.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8322024-05-30T04:31:02.800ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8332024-05-30T04:31:02.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8342024-05-30T04:31:02.801ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8352024-05-30T04:31:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8362024-05-30T04:31:02.808ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 671.314324ms iteration = 7
8372024-05-30T04:31:08.716ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 774 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 6f5dafd4-2645-4fb4-bd98-6385f6668c57 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
8382024-05-30T04:31:12.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
8392024-05-30T04:31:12.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8402024-05-30T04:31:12.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8412024-05-30T04:31:12.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8422024-05-30T04:31:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8432024-05-30T04:31:12.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8442024-05-30T04:31:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8452024-05-30T04:31:12.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8462024-05-30T04:31:12.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8472024-05-30T04:31:12.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8482024-05-30T04:31:12.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
8492024-05-30T04:31:12.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8502024-05-30T04:31:12.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8512024-05-30T04:31:12.153ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
8522024-05-30T04:31:12.153ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.747791ms iteration = 20
8532024-05-30T04:31:18.716ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 412 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 2239c292-674d-49a5-bb52-fb1a05ee2c25 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
8542024-05-30T04:31:19.379ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:54519
8552024-05-30T04:31:19.379ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54519 req_id = dbaccb27-2d4c-4aa6-b3e2-8fdbe37dcb2e roles = RoleSet { roles: {} } uri = /metrics/producers
8562024-05-30T04:31:19.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54519 req_id = dbaccb27-2d4c-4aa6-b3e2-8fdbe37dcb2e resource = Database result = Ok(()) uri = /metrics/producers
8572024-05-30T04:31:19.411ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
8582024-05-30T04:31:19.411ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54519 req_id = dbaccb27-2d4c-4aa6-b3e2-8fdbe37dcb2e roles = RoleSet { roles: {} } uri = /metrics/producers
8592024-05-30T04:31:19.411ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54519 req_id = dbaccb27-2d4c-4aa6-b3e2-8fdbe37dcb2e resource = Database result = Ok(()) uri = /metrics/producers
8602024-05-30T04:31:19.415ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
8612024-05-30T04:31:19.416ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "65d51f16-f543-4b19-8df9-770d0adc2b57", "date": "Thu, 30 May 2024 04:31:19 GMT"} })
8622024-05-30T04:31:19.416ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 439e757e-803c-4f81-8f3d-8bd0f1f5d2cc
8632024-05-30T04:31:19.418ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 38772 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:54519 req_id = dbaccb27-2d4c-4aa6-b3e2-8fdbe37dcb2e response_code = 201 uri = /metrics/producers
8642024-05-30T04:31:22.137ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
8652024-05-30T04:31:22.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8662024-05-30T04:31:22.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8672024-05-30T04:31:22.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8682024-05-30T04:31:22.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8692024-05-30T04:31:22.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8702024-05-30T04:31:22.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8712024-05-30T04:31:22.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8722024-05-30T04:31:22.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8732024-05-30T04:31:22.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8742024-05-30T04:31:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
8752024-05-30T04:31:22.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8762024-05-30T04:31:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8772024-05-30T04:31:22.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
8782024-05-30T04:31:22.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.103334ms iteration = 21
8792024-05-30T04:31:28.717ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 656 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = d83d1f21-8ee3-4a4e-998b-9afceb07c966 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
8802024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
8812024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8822024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
8832024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
8842024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8852024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8862024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
8872024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
8882024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8892024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8902024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
8912024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
8922024-05-30T04:31:32.138ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8932024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8942024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
8952024-05-30T04:31:32.138ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
8962024-05-30T04:31:32.138ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
8972024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
8982024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.9\u{b5}s iteration = 8
8992024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
9002024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9012024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9022024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9032024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9042024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9052024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9062024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9072024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9082024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9092024-05-30T04:31:32.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9102024-05-30T04:31:32.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9112024-05-30T04:31:32.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9122024-05-30T04:31:32.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9132024-05-30T04:31:32.141ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1114, tv_nsec: 255093001 } })
9142024-05-30T04:31:32.141ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
9152024-05-30T04:31:32.141ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.241057ms iteration = 8
9162024-05-30T04:31:32.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9172024-05-30T04:31:32.142ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9182024-05-30T04:31:32.142ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
9192024-05-30T04:31:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
9202024-05-30T04:31:32.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9212024-05-30T04:31:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9222024-05-30T04:31:32.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1114, tv_nsec: 258067485 } })
9232024-05-30T04:31:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9242024-05-30T04:31:32.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9252024-05-30T04:31:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9262024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9272024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9282024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9292024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9302024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9312024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9322024-05-30T04:31:32.145ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
9332024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9342024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9352024-05-30T04:31:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9362024-05-30T04:31:32.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9372024-05-30T04:31:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9382024-05-30T04:31:32.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9392024-05-30T04:31:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9402024-05-30T04:31:32.147ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
9412024-05-30T04:31:32.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9422024-05-30T04:31:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9432024-05-30T04:31:32.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1114, tv_nsec: 261641726 } })
9442024-05-30T04:31:32.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452024-05-30T04:31:32.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9462024-05-30T04:31:32.148ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
9472024-05-30T04:31:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
9482024-05-30T04:31:32.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9492024-05-30T04:31:32.150ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
9502024-05-30T04:31:32.150ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.345192ms iteration = 8
9512024-05-30T04:31:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9522024-05-30T04:31:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
9532024-05-30T04:31:32.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9542024-05-30T04:31:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9552024-05-30T04:31:32.156ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9562024-05-30T04:31:32.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9572024-05-30T04:31:32.157ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9582024-05-30T04:31:32.158ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
9592024-05-30T04:31:32.158ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.175582ms iteration = 9
9602024-05-30T04:31:32.158ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9612024-05-30T04:31:32.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9622024-05-30T04:31:32.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
9632024-05-30T04:31:32.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.034962ms iteration = 22
9642024-05-30T04:31:32.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9652024-05-30T04:31:32.159ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9662024-05-30T04:31:32.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9672024-05-30T04:31:32.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9682024-05-30T04:31:32.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9692024-05-30T04:31:32.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9702024-05-30T04:31:32.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9712024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9722024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9732024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9742024-05-30T04:31:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
9752024-05-30T04:31:32.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9762024-05-30T04:31:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9772024-05-30T04:31:32.166ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.642488ms iteration = 8
9782024-05-30T04:31:32.245ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
9792024-05-30T04:31:32.245ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9802024-05-30T04:31:32.245ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f378fc58-01ef-4364-bfa9-8f77737c439d", "content-length": "26", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9812024-05-30T04:31:32.245ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9822024-05-30T04:31:32.247ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9832024-05-30T04:31:32.248ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
9842024-05-30T04:31:32.248ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9852024-05-30T04:31:32.248ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ab7ccfc6-2550-4830-b250-7b66788fd6f6", "content-length": "26", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9862024-05-30T04:31:32.248ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9872024-05-30T04:31:32.249ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9882024-05-30T04:31:32.264ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
9892024-05-30T04:31:32.264ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9902024-05-30T04:31:32.265ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1bd378e0-7e8b-4aa9-8015-4ef3c7a24d90", "content-length": "26", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9912024-05-30T04:31:32.265ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9922024-05-30T04:31:32.267ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
9932024-05-30T04:31:32.278ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9942024-05-30T04:31:32.278ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "84a0be47-2c8f-43a9-9759-3bb791a5388f", "content-length": "691", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9952024-05-30T04:31:32.278ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
9962024-05-30T04:31:32.278ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
9972024-05-30T04:31:32.278ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
9982024-05-30T04:31:32.280ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 141.947193ms iteration = 8
9992024-05-30T04:31:32.335ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
10002024-05-30T04:31:32.335ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dd1b0cb2-e077-4819-bc0c-6d81b0f766e8", "content-length": "1", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
10012024-05-30T04:31:32.337ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10022024-05-30T04:31:32.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10032024-05-30T04:31:32.339ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 201.055113ms iteration = 8
10042024-05-30T04:31:32.390ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 252.02463ms iteration = 8
10052024-05-30T04:31:32.459ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10062024-05-30T04:31:32.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10072024-05-30T04:31:32.461ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
10082024-05-30T04:31:32.461ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10092024-05-30T04:31:32.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10102024-05-30T04:31:32.469ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
10112024-05-30T04:31:32.469ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85e3943c-74ca-4ab3-bcd1-53b9ad6c760b", "content-length": "121", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
10122024-05-30T04:31:32.469ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
10132024-05-30T04:31:32.469ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
10142024-05-30T04:31:32.470ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
10152024-05-30T04:31:32.470ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
10162024-05-30T04:31:32.470ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
10172024-05-30T04:31:32.470ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
10182024-05-30T04:31:32.470ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
10192024-05-30T04:31:32.470ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
10202024-05-30T04:31:32.470ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
10212024-05-30T04:31:32.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10222024-05-30T04:31:32.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10232024-05-30T04:31:32.471ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
10242024-05-30T04:31:32.472ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fa0b898a-3b1b-40de-8cd3-ff32c479e258", "content-length": "2", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
10252024-05-30T04:31:32.472ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
10262024-05-30T04:31:32.472ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "db807497-ff9f-4611-9157-ba6f7231267c", "content-length": "52", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
10272024-05-30T04:31:32.472ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
10282024-05-30T04:31:32.472ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
10292024-05-30T04:31:32.564ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6155a6d0-5a95-45f8-ae9e-ef9f3c228fd9", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
10302024-05-30T04:31:32.564ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
10312024-05-30T04:31:32.564ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
10322024-05-30T04:31:32.564ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "257cdb9b-44ef-4f66-b9cd-56545463bbd8", "content-length": "467", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
10332024-05-30T04:31:32.564ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10342024-05-30T04:31:32.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10352024-05-30T04:31:32.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10362024-05-30T04:31:32.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10372024-05-30T04:31:32.569ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10382024-05-30T04:31:32.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10392024-05-30T04:31:32.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10402024-05-30T04:31:32.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10412024-05-30T04:31:32.577ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 438.91565ms iteration = 8
10422024-05-30T04:31:38.718ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 585 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = af5d263f-bc89-415d-9df8-7a33966e321c response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
10432024-05-30T04:31:42.138ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
10442024-05-30T04:31:42.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10452024-05-30T04:31:42.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10462024-05-30T04:31:42.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10472024-05-30T04:31:42.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10482024-05-30T04:31:42.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10492024-05-30T04:31:42.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10502024-05-30T04:31:42.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10512024-05-30T04:31:42.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10522024-05-30T04:31:42.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10532024-05-30T04:31:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
10542024-05-30T04:31:42.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10552024-05-30T04:31:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10562024-05-30T04:31:42.155ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
10572024-05-30T04:31:42.155ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.558282ms iteration = 23
10582024-05-30T04:31:48.718ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1258 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 30c0cca5-950b-4766-a8c5-14c5bc0bfff3 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
10592024-05-30T04:31:52.138ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
10602024-05-30T04:31:52.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10612024-05-30T04:31:52.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10622024-05-30T04:31:52.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10632024-05-30T04:31:52.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10642024-05-30T04:31:52.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10652024-05-30T04:31:52.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10662024-05-30T04:31:52.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10672024-05-30T04:31:52.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10682024-05-30T04:31:52.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10692024-05-30T04:31:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
10702024-05-30T04:31:52.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10712024-05-30T04:31:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10722024-05-30T04:31:52.154ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
10732024-05-30T04:31:52.154ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.921263ms iteration = 24
10742024-05-30T04:31:58.718ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 727 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 0bd31e61-112b-4c33-9fcc-14fb5df4e050 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
10752024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
10762024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10772024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
10782024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
10792024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10802024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
10812024-05-30T04:32:02.138ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
10822024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
10832024-05-30T04:32:02.138ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
10842024-05-30T04:32:02.138ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
10852024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.72\u{b5}s iteration = 9
10862024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
10872024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
10882024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10892024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10902024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
10912024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
10922024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10932024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
10942024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10952024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10962024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
10972024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10982024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
10992024-05-30T04:32:02.138ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:22:02.138632201 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
11002024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
11012024-05-30T04:32:02.138ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
11022024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
11032024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
11042024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
11052024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11062024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
11072024-05-30T04:32:02.139ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
11082024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.07\u{b5}s iteration = 6
11092024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
11102024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
11112024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11122024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
11132024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
11142024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11152024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11162024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11172024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
11182024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11192024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11202024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11212024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11222024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11232024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11242024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
11252024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11262024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11272024-05-30T04:32:02.139ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 252683543 } })
11282024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11292024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11302024-05-30T04:32:02.139ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11312024-05-30T04:32:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11322024-05-30T04:32:02.140ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 253616744 } })
11332024-05-30T04:32:02.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11342024-05-30T04:32:02.141ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
11352024-05-30T04:32:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.57927ms iteration = 6
11362024-05-30T04:32:02.141ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
11372024-05-30T04:32:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.615341ms iteration = 6
11382024-05-30T04:32:02.141ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
11392024-05-30T04:32:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.318758ms iteration = 9
11402024-05-30T04:32:02.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11412024-05-30T04:32:02.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11422024-05-30T04:32:02.142ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11432024-05-30T04:32:02.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11442024-05-30T04:32:02.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11452024-05-30T04:32:02.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 256701200 } })
11462024-05-30T04:32:02.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11472024-05-30T04:32:02.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 256872571 } })
11482024-05-30T04:32:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11492024-05-30T04:32:02.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11502024-05-30T04:32:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11512024-05-30T04:32:02.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
11522024-05-30T04:32:02.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
11532024-05-30T04:32:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11542024-05-30T04:32:02.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11552024-05-30T04:32:02.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11562024-05-30T04:32:02.145ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
11572024-05-30T04:32:02.145ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
11582024-05-30T04:32:02.145ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
11592024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11602024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
11612024-05-30T04:32:02.146ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:22:02.138632201 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
11622024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.672998ms iteration = 5
11632024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11642024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11652024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11662024-05-30T04:32:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11672024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11682024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11692024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11702024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11712024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11722024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11732024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11742024-05-30T04:32:02.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11752024-05-30T04:32:02.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 260922678 } })
11762024-05-30T04:32:02.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
11772024-05-30T04:32:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11782024-05-30T04:32:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11792024-05-30T04:32:02.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11802024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11812024-05-30T04:32:02.149ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
11822024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11832024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11842024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
11852024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
11862024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
11872024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11882024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11892024-05-30T04:32:02.149ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
11902024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
11912024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
11922024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11932024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11942024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11952024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11962024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972024-05-30T04:32:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11982024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11992024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
12012024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12022024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12032024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12042024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12052024-05-30T04:32:02.153ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12062024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12072024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12082024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12092024-05-30T04:32:02.154ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
12102024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 16.256767ms iteration = 9
12112024-05-30T04:32:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12122024-05-30T04:32:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12132024-05-30T04:32:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12142024-05-30T04:32:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12152024-05-30T04:32:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12162024-05-30T04:32:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12172024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
12182024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12192024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12202024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
12212024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12222024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12232024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
12242024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
12252024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12262024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12272024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12282024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12292024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12302024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12312024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12322024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12332024-05-30T04:32:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12342024-05-30T04:32:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12352024-05-30T04:32:02.161ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12362024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
12372024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.506462ms iteration = 25
12382024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12392024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12402024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12412024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
12422024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
12432024-05-30T04:32:02.164ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
12442024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 25.841437ms iteration = 10
12452024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
12462024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
12472024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
12482024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
12492024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12502024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12512024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12522024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12532024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
12542024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
12552024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
12562024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.688199ms iteration = 6
12572024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
12582024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
12592024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
12602024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 27.966582ms iteration = 6
12612024-05-30T04:32:02.167ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12622024-05-30T04:32:02.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12632024-05-30T04:32:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12642024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12652024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12662024-05-30T04:32:02.183ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12672024-05-30T04:32:02.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12682024-05-30T04:32:02.184ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12692024-05-30T04:32:02.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
12702024-05-30T04:32:02.187ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12712024-05-30T04:32:02.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12722024-05-30T04:32:02.189ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 50.790095ms iteration = 9
12732024-05-30T04:32:02.207ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
12742024-05-30T04:32:02.207ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
12752024-05-30T04:32:02.207ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 62.037164ms iteration = 5
12762024-05-30T04:32:02.238ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
12772024-05-30T04:32:02.238ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 99.802568ms iteration = 6
12782024-05-30T04:32:02.279ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12792024-05-30T04:32:02.297ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12802024-05-30T04:32:02.312ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
12812024-05-30T04:32:02.312ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12822024-05-30T04:32:02.313ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a032fa22-43e0-4004-b80f-137d443c82b7", "content-length": "26", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12832024-05-30T04:32:02.313ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12842024-05-30T04:32:02.314ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12852024-05-30T04:32:02.316ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
12862024-05-30T04:32:02.316ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12872024-05-30T04:32:02.316ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e445cd96-bbbf-4744-9a56-e0c283983b74", "content-length": "26", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12882024-05-30T04:32:02.316ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12892024-05-30T04:32:02.318ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12902024-05-30T04:32:02.334ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
12912024-05-30T04:32:02.334ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12922024-05-30T04:32:02.335ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b4724ca6-d120-4a32-86d0-3ac8dcb90199", "content-length": "26", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12932024-05-30T04:32:02.335ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12942024-05-30T04:32:02.336ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
12952024-05-30T04:32:02.350ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12962024-05-30T04:32:02.350ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2fb11011-a965-4111-b196-21edca6d0f74", "content-length": "691", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12972024-05-30T04:32:02.350ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
12982024-05-30T04:32:02.350ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
12992024-05-30T04:32:02.350ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
13002024-05-30T04:32:02.352ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 213.954312ms iteration = 9
13012024-05-30T04:32:02.434ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13022024-05-30T04:32:02.462ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13032024-05-30T04:32:02.464ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "71c35fae-620e-480e-b98d-a4d0638d6aa0", "content-length": "1", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
13042024-05-30T04:32:02.465ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13052024-05-30T04:32:02.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13062024-05-30T04:32:02.467ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 329.250228ms iteration = 9
13072024-05-30T04:32:02.473ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 334.789202ms iteration = 9
13082024-05-30T04:32:02.536ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13092024-05-30T04:32:02.536ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1e4028b9-2d0e-4eab-b2ba-30a0ac9bb6df", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13102024-05-30T04:32:02.536ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13112024-05-30T04:32:02.538ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c8097fab-d444-4c13-9b78-b2674526993a", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13122024-05-30T04:32:02.538ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13132024-05-30T04:32:02.540ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13142024-05-30T04:32:02.540ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 401.723492ms iteration = 7
13152024-05-30T04:32:02.547ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13162024-05-30T04:32:02.547ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0f440bf9-911b-47e1-b72d-0f10e16f0eba", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13172024-05-30T04:32:02.548ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13182024-05-30T04:32:02.549ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "97a3bc47-b4d9-4975-bd5e-6bbf433a0f91", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13192024-05-30T04:32:02.549ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13202024-05-30T04:32:02.550ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "62afc12e-c46b-4894-bbee-c94824b71178", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13212024-05-30T04:32:02.550ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13222024-05-30T04:32:02.552ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13232024-05-30T04:32:02.552ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 413.623729ms iteration = 7
13242024-05-30T04:32:02.612ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13252024-05-30T04:32:02.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13262024-05-30T04:32:02.614ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
13272024-05-30T04:32:02.614ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13282024-05-30T04:32:02.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13292024-05-30T04:32:02.622ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13302024-05-30T04:32:02.622ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d83b52d8-6ac3-4bc4-87f4-02370e2c097b", "content-length": "121", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
13312024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13322024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
13332024-05-30T04:32:02.622ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
13342024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
13352024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
13362024-05-30T04:32:02.622ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
13372024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
13382024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
13392024-05-30T04:32:02.622ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
13402024-05-30T04:32:02.622ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13412024-05-30T04:32:02.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13422024-05-30T04:32:02.624ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13432024-05-30T04:32:02.624ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "89720fce-8357-492b-bde5-090048e63a97", "content-length": "2", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
13442024-05-30T04:32:02.624ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13452024-05-30T04:32:02.624ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7ad09e93-7d31-4750-91de-5dbd2f4b4572", "content-length": "52", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
13462024-05-30T04:32:02.624ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
13472024-05-30T04:32:02.624ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13482024-05-30T04:32:02.677ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "72368fd6-3293-492a-9635-d096dbd49dc4", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
13492024-05-30T04:32:02.677ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
13502024-05-30T04:32:02.677ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13512024-05-30T04:32:02.677ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c6e4f868-9f98-4de2-b0ef-775c8c636ec4", "content-length": "467", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
13522024-05-30T04:32:02.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13532024-05-30T04:32:02.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13542024-05-30T04:32:02.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13552024-05-30T04:32:02.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13562024-05-30T04:32:02.683ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13572024-05-30T04:32:02.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13582024-05-30T04:32:02.685ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13592024-05-30T04:32:02.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13602024-05-30T04:32:02.692ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 553.895102ms iteration = 9
13612024-05-30T04:32:08.718ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 493 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = a4ddfa98-0318-485e-aeca-28fc1d7eb9c9 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
13622024-05-30T04:32:08.926ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:41644
13632024-05-30T04:32:08.927ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41644 req_id = 6c4bb336-b3d1-4ae0-9167-68663b13059c roles = RoleSet { roles: {} } uri = /metrics/producers
13642024-05-30T04:32:08.927ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41644 req_id = 6c4bb336-b3d1-4ae0-9167-68663b13059c resource = Database result = Ok(()) uri = /metrics/producers
13652024-05-30T04:32:08.960ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
13662024-05-30T04:32:08.960ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41644 req_id = 6c4bb336-b3d1-4ae0-9167-68663b13059c roles = RoleSet { roles: {} } uri = /metrics/producers
13672024-05-30T04:32:08.960ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41644 req_id = 6c4bb336-b3d1-4ae0-9167-68663b13059c resource = Database result = Ok(()) uri = /metrics/producers
13682024-05-30T04:32:08.988ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
13692024-05-30T04:32:08.989ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7cedc0a7-d3b0-48ac-a86f-84ce982d5060", "date": "Thu, 30 May 2024 04:32:08 GMT"} })
13702024-05-30T04:32:08.989ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 0cf65393-f6a2-480b-8f0b-78c9543ba992
13712024-05-30T04:32:08.991ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 64742 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41644 req_id = 6c4bb336-b3d1-4ae0-9167-68663b13059c response_code = 201 uri = /metrics/producers
13722024-05-30T04:32:12.138ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
13732024-05-30T04:32:12.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13742024-05-30T04:32:12.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13752024-05-30T04:32:12.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13762024-05-30T04:32:12.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13772024-05-30T04:32:12.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13782024-05-30T04:32:12.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13792024-05-30T04:32:12.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13802024-05-30T04:32:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13812024-05-30T04:32:12.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13822024-05-30T04:32:12.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
13832024-05-30T04:32:12.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13842024-05-30T04:32:12.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13852024-05-30T04:32:12.156ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
13862024-05-30T04:32:12.156ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.690033ms iteration = 26
13872024-05-30T04:32:18.717ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 466 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = dcc230a7-38f7-4ac2-88e0-9ff0b9d2936f response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
13882024-05-30T04:32:22.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
13892024-05-30T04:32:22.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13902024-05-30T04:32:22.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13912024-05-30T04:32:22.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13922024-05-30T04:32:22.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13932024-05-30T04:32:22.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13942024-05-30T04:32:22.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13952024-05-30T04:32:22.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13962024-05-30T04:32:22.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13972024-05-30T04:32:22.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13982024-05-30T04:32:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
13992024-05-30T04:32:22.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14002024-05-30T04:32:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14012024-05-30T04:32:22.155ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
14022024-05-30T04:32:22.155ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.819493ms iteration = 27
14032024-05-30T04:32:28.719ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 969 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = a1ee8ce6-e998-4e5e-a2c2-ca0aac7d24b8 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
14042024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
14052024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14062024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
14072024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
14082024-05-30T04:32:32.139ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
14092024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14102024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
14112024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
14122024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14132024-05-30T04:32:32.139ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
14142024-05-30T04:32:32.139ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
14152024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 16.77\u{b5}s iteration = 10
14162024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
14172024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
14182024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
14192024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14202024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14212024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14222024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
14232024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14242024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
14252024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14262024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14272024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14282024-05-30T04:32:32.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14292024-05-30T04:32:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14302024-05-30T04:32:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14312024-05-30T04:32:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14322024-05-30T04:32:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14332024-05-30T04:32:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14342024-05-30T04:32:32.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14352024-05-30T04:32:32.142ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14362024-05-30T04:32:32.142ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
14372024-05-30T04:32:32.142ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.885573ms iteration = 10
14382024-05-30T04:32:32.142ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1174, tv_nsec: 255450205 } })
14392024-05-30T04:32:32.142ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1174, tv_nsec: 255586957 } })
14402024-05-30T04:32:32.142ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14412024-05-30T04:32:32.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14422024-05-30T04:32:32.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
14432024-05-30T04:32:32.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14442024-05-30T04:32:32.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
14452024-05-30T04:32:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14462024-05-30T04:32:32.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14472024-05-30T04:32:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14482024-05-30T04:32:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14492024-05-30T04:32:32.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14502024-05-30T04:32:32.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14512024-05-30T04:32:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
14522024-05-30T04:32:32.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
14532024-05-30T04:32:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
14542024-05-30T04:32:32.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1174, tv_nsec: 259828335 } })
14552024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
14562024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14572024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14582024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14592024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14602024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14612024-05-30T04:32:32.147ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
14622024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14632024-05-30T04:32:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14642024-05-30T04:32:32.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
14652024-05-30T04:32:32.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14662024-05-30T04:32:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14672024-05-30T04:32:32.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14682024-05-30T04:32:32.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14692024-05-30T04:32:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14702024-05-30T04:32:32.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712024-05-30T04:32:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14722024-05-30T04:32:32.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14732024-05-30T04:32:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14742024-05-30T04:32:32.152ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
14752024-05-30T04:32:32.152ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.01571ms iteration = 10
14762024-05-30T04:32:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
14772024-05-30T04:32:32.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14782024-05-30T04:32:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14792024-05-30T04:32:32.157ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14802024-05-30T04:32:32.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14812024-05-30T04:32:32.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14822024-05-30T04:32:32.158ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
14832024-05-30T04:32:32.158ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.380993ms iteration = 11
14842024-05-30T04:32:32.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
14852024-05-30T04:32:32.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.185222ms iteration = 28
14862024-05-30T04:32:32.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14872024-05-30T04:32:32.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14882024-05-30T04:32:32.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14892024-05-30T04:32:32.160ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14902024-05-30T04:32:32.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14912024-05-30T04:32:32.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14922024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14932024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14942024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14952024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14962024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14972024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14982024-05-30T04:32:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
14992024-05-30T04:32:32.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15002024-05-30T04:32:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15012024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.75237ms iteration = 10
15022024-05-30T04:32:32.246ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
15032024-05-30T04:32:32.246ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15042024-05-30T04:32:32.246ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e00cc617-c528-47d8-891e-57c6ec719a8d", "content-length": "26", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15052024-05-30T04:32:32.246ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15062024-05-30T04:32:32.247ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15072024-05-30T04:32:32.252ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
15082024-05-30T04:32:32.252ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15092024-05-30T04:32:32.252ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6728b7ff-3a47-4e01-a71d-e43fdb36f3b7", "content-length": "26", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15102024-05-30T04:32:32.252ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15112024-05-30T04:32:32.253ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
15122024-05-30T04:32:32.253ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15132024-05-30T04:32:32.253ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
15142024-05-30T04:32:32.253ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6051bebd-89be-4a2f-a686-6e99c235bff7", "content-length": "26", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15152024-05-30T04:32:32.253ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15162024-05-30T04:32:32.255ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15172024-05-30T04:32:32.277ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15182024-05-30T04:32:32.278ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a534db8d-a856-4a58-bfd5-796d5603f698", "content-length": "691", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15192024-05-30T04:32:32.278ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
15202024-05-30T04:32:32.278ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
15212024-05-30T04:32:32.278ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
15222024-05-30T04:32:32.280ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 140.7956ms iteration = 10
15232024-05-30T04:32:32.336ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 196.563012ms iteration = 10
15242024-05-30T04:32:32.336ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
15252024-05-30T04:32:32.336ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3e0b8746-a17e-4d64-b59f-04cefad741b1", "content-length": "1", "date": "Thu, 30 May 2024 04:32:31 GMT"} })
15262024-05-30T04:32:32.337ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15272024-05-30T04:32:32.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15282024-05-30T04:32:32.339ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 199.610707ms iteration = 10
15292024-05-30T04:32:32.396ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15302024-05-30T04:32:32.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15312024-05-30T04:32:32.398ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
15322024-05-30T04:32:32.398ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15332024-05-30T04:32:32.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15342024-05-30T04:32:32.404ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
15352024-05-30T04:32:32.404ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8be201df-3858-4005-88c0-e9b8087a9c9e", "content-length": "121", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15362024-05-30T04:32:32.404ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
15372024-05-30T04:32:32.404ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
15382024-05-30T04:32:32.405ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
15392024-05-30T04:32:32.405ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
15402024-05-30T04:32:32.405ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
15412024-05-30T04:32:32.405ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
15422024-05-30T04:32:32.405ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
15432024-05-30T04:32:32.405ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
15442024-05-30T04:32:32.405ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
15452024-05-30T04:32:32.405ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15462024-05-30T04:32:32.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15472024-05-30T04:32:32.407ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
15482024-05-30T04:32:32.407ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9d28d56a-1b83-417f-8d1f-34389af83989", "content-length": "2", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15492024-05-30T04:32:32.407ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
15502024-05-30T04:32:32.407ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e8f3d581-c432-46b6-95d5-f4525518656d", "content-length": "52", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15512024-05-30T04:32:32.407ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
15522024-05-30T04:32:32.407ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
15532024-05-30T04:32:32.490ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "67cee4ca-78f1-4e8b-9841-1a8319d372be", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15542024-05-30T04:32:32.490ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
15552024-05-30T04:32:32.490ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15562024-05-30T04:32:32.490ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4caad78b-c4ad-4924-8609-2a4cf931e361", "content-length": "467", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
15572024-05-30T04:32:32.490ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15582024-05-30T04:32:32.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15592024-05-30T04:32:32.493ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15602024-05-30T04:32:32.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15612024-05-30T04:32:32.495ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15622024-05-30T04:32:32.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15632024-05-30T04:32:32.496ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15642024-05-30T04:32:32.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15652024-05-30T04:32:32.503ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 363.607924ms iteration = 10
15662024-05-30T04:32:38.717ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 431 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = d8d22684-33df-492d-855a-877f3ab254a4 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
15672024-05-30T04:32:42.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
15682024-05-30T04:32:42.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15692024-05-30T04:32:42.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15702024-05-30T04:32:42.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15712024-05-30T04:32:42.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15722024-05-30T04:32:42.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15732024-05-30T04:32:42.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15742024-05-30T04:32:42.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15752024-05-30T04:32:42.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15762024-05-30T04:32:42.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15772024-05-30T04:32:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
15782024-05-30T04:32:42.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15792024-05-30T04:32:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15802024-05-30T04:32:42.156ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
15812024-05-30T04:32:42.156ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.987706ms iteration = 29
15822024-05-30T04:32:48.718ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 435 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 66069945-6d41-416c-93ab-70b9c0904925 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
15832024-05-30T04:32:52.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
15842024-05-30T04:32:52.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15852024-05-30T04:32:52.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15862024-05-30T04:32:52.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15872024-05-30T04:32:52.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15882024-05-30T04:32:52.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15892024-05-30T04:32:52.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15902024-05-30T04:32:52.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15912024-05-30T04:32:52.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15922024-05-30T04:32:52.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15932024-05-30T04:32:52.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
15942024-05-30T04:32:52.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15952024-05-30T04:32:52.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15962024-05-30T04:32:52.156ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
15972024-05-30T04:32:52.156ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.783263ms iteration = 30
15982024-05-30T04:32:58.718ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 490 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 150a03fa-7555-4671-ba77-0a5fdfa1b8c8 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
15992024-05-30T04:33:02.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
16002024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
16012024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
16022024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16032024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16042024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
16052024-05-30T04:33:02.140ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:23:02.139978295 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
16062024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
16072024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16082024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
16092024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16102024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
16112024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16122024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16132024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
16142024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16152024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
16162024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16172024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
16182024-05-30T04:33:02.140ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
16192024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
16202024-05-30T04:33:02.140ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
16212024-05-30T04:33:02.140ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
16222024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
16232024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.851\u{b5}s iteration = 11
16242024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
16252024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
16262024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16272024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16282024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
16292024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
16302024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
16312024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16322024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
16332024-05-30T04:33:02.140ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
16342024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
16352024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.701\u{b5}s iteration = 7
16362024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16372024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
16382024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16392024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16402024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16412024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16422024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16432024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
16442024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16452024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16462024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16472024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16482024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16492024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16502024-05-30T04:33:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16512024-05-30T04:33:02.140ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 253184289 } })
16522024-05-30T04:33:02.140ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 253229670 } })
16532024-05-30T04:33:02.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16542024-05-30T04:33:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16552024-05-30T04:33:02.142ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
16562024-05-30T04:33:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.563809ms iteration = 7
16572024-05-30T04:33:02.142ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
16582024-05-30T04:33:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.63687ms iteration = 7
16592024-05-30T04:33:02.142ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16602024-05-30T04:33:02.143ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16612024-05-30T04:33:02.143ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16622024-05-30T04:33:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16632024-05-30T04:33:02.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16642024-05-30T04:33:02.143ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
16652024-05-30T04:33:02.143ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.636592ms iteration = 11
16662024-05-30T04:33:02.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 256168543 } })
16672024-05-30T04:33:02.143ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:23:02.139978295 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
16682024-05-30T04:33:02.143ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.003956ms iteration = 6
16692024-05-30T04:33:02.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 256331695 } })
16702024-05-30T04:33:02.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16712024-05-30T04:33:02.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
16722024-05-30T04:33:02.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16732024-05-30T04:33:02.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
16742024-05-30T04:33:02.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16752024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16762024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16772024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16782024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16792024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16802024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
16812024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
16822024-05-30T04:33:02.146ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
16832024-05-30T04:33:02.146ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
16842024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16852024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
16862024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16872024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16882024-05-30T04:33:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
16892024-05-30T04:33:02.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16902024-05-30T04:33:02.147ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16912024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16922024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16932024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16942024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16952024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16962024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16972024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16982024-05-30T04:33:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16992024-05-30T04:33:02.149ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17002024-05-30T04:33:02.149ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17012024-05-30T04:33:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17022024-05-30T04:33:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17032024-05-30T04:33:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17042024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17052024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17062024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17072024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17082024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17092024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
17102024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
17112024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
17122024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17132024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17142024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17152024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17162024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17172024-05-30T04:33:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17182024-05-30T04:33:02.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 264219166 } })
17192024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17202024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17212024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17222024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17232024-05-30T04:33:02.152ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
17242024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17252024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17262024-05-30T04:33:02.152ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
17272024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17282024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17292024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
17302024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
17312024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
17322024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17332024-05-30T04:33:02.153ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
17342024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.263203ms iteration = 11
17352024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17362024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17372024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17382024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17392024-05-30T04:33:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
17402024-05-30T04:33:02.154ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17412024-05-30T04:33:02.154ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17422024-05-30T04:33:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17432024-05-30T04:33:02.155ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17442024-05-30T04:33:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17452024-05-30T04:33:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17462024-05-30T04:33:02.156ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17472024-05-30T04:33:02.156ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17482024-05-30T04:33:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17492024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
17502024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17512024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17522024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17532024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17542024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17552024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17562024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17572024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17582024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17592024-05-30T04:33:02.160ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
17602024-05-30T04:33:02.160ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
17612024-05-30T04:33:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17622024-05-30T04:33:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17632024-05-30T04:33:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17642024-05-30T04:33:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17652024-05-30T04:33:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17662024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17672024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17682024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17692024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17702024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17712024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17722024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17732024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17742024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
17752024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
17762024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17772024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17782024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17792024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17802024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17812024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17822024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17832024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17842024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17852024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17862024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17872024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17882024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17892024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17902024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
17912024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.840241ms iteration = 31
17922024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17932024-05-30T04:33:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17942024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17952024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17962024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17972024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17982024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
17992024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
18002024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
18012024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 29.842183ms iteration = 7
18022024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18032024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18042024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18052024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18062024-05-30T04:33:02.170ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
18072024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 30.259678ms iteration = 12
18082024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18092024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
18102024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
18112024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
18122024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 32.15755ms iteration = 7
18132024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
18142024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18152024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
18162024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18172024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18182024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18192024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
18202024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18212024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18222024-05-30T04:33:02.176ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.788223ms iteration = 11
18232024-05-30T04:33:02.178ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18242024-05-30T04:33:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18252024-05-30T04:33:02.185ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18262024-05-30T04:33:02.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18272024-05-30T04:33:02.189ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18282024-05-30T04:33:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18292024-05-30T04:33:02.193ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18302024-05-30T04:33:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18312024-05-30T04:33:02.195ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18322024-05-30T04:33:02.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18332024-05-30T04:33:02.196ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18342024-05-30T04:33:02.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18352024-05-30T04:33:02.198ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18362024-05-30T04:33:02.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
18372024-05-30T04:33:02.204ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
18382024-05-30T04:33:02.204ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
18392024-05-30T04:33:02.204ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 57.707034ms iteration = 6
18402024-05-30T04:33:02.204ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18412024-05-30T04:33:02.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18422024-05-30T04:33:02.208ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18432024-05-30T04:33:02.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18442024-05-30T04:33:02.209ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18452024-05-30T04:33:02.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18462024-05-30T04:33:02.211ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18472024-05-30T04:33:02.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18482024-05-30T04:33:02.213ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18492024-05-30T04:33:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18502024-05-30T04:33:02.215ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18512024-05-30T04:33:02.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18522024-05-30T04:33:02.216ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18532024-05-30T04:33:02.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18542024-05-30T04:33:02.218ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18552024-05-30T04:33:02.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18562024-05-30T04:33:02.221ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18572024-05-30T04:33:02.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18582024-05-30T04:33:02.223ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18592024-05-30T04:33:02.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18602024-05-30T04:33:02.224ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18612024-05-30T04:33:02.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18622024-05-30T04:33:02.225ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18632024-05-30T04:33:02.226ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
18642024-05-30T04:33:02.226ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 86.153991ms iteration = 7
18652024-05-30T04:33:02.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18662024-05-30T04:33:02.229ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18672024-05-30T04:33:02.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18682024-05-30T04:33:02.313ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
18692024-05-30T04:33:02.315ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
18702024-05-30T04:33:02.322ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
18712024-05-30T04:33:02.322ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18722024-05-30T04:33:02.322ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5c6010c8-58de-441f-984d-3bd113c97c97", "content-length": "26", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18732024-05-30T04:33:02.322ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18742024-05-30T04:33:02.324ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18752024-05-30T04:33:02.341ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
18762024-05-30T04:33:02.341ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18772024-05-30T04:33:02.341ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "aae97c7f-e440-4b7b-a51d-58b90ca7d1ee", "content-length": "26", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18782024-05-30T04:33:02.341ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18792024-05-30T04:33:02.342ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
18802024-05-30T04:33:02.398ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18812024-05-30T04:33:02.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18822024-05-30T04:33:02.399ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18832024-05-30T04:33:02.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18842024-05-30T04:33:02.401ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18852024-05-30T04:33:02.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18862024-05-30T04:33:02.403ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18872024-05-30T04:33:02.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18882024-05-30T04:33:02.405ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18892024-05-30T04:33:02.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18902024-05-30T04:33:02.406ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18912024-05-30T04:33:02.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18922024-05-30T04:33:02.407ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18932024-05-30T04:33:02.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18942024-05-30T04:33:02.408ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
18952024-05-30T04:33:02.408ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18962024-05-30T04:33:02.408ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ce293f89-d01f-4ac1-a507-8f30e36d10a2", "content-length": "26", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18972024-05-30T04:33:02.408ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18982024-05-30T04:33:02.409ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18992024-05-30T04:33:02.410ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19002024-05-30T04:33:02.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
19012024-05-30T04:33:02.413ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19022024-05-30T04:33:02.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19032024-05-30T04:33:02.415ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19042024-05-30T04:33:02.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19052024-05-30T04:33:02.416ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19062024-05-30T04:33:02.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19072024-05-30T04:33:02.418ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19082024-05-30T04:33:02.421ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
19092024-05-30T04:33:02.421ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19102024-05-30T04:33:02.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19112024-05-30T04:33:02.427ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: e232750d-d52e-4ac4-8c87-d7b0e4145a1f, kind: Service { id: 8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3 }, name: Name("external-dns-8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 109])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: fb84270c-485b-4df1-a4a3-7bf184a5f27c, kind: Service { id: f72d0bad-b96e-4cb4-9157-126b67797b9b }, name: Name("external-dns-f72d0bad-b96e-4cb4-9157-126b67797b9b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 1fbd52c8-c004-450c-ad40-e669ea7ac97a, kind: Service { id: b5bf7e90-33d3-4f67-b9d3-793143654e9a }, name: Name("nexus-b5bf7e90-33d3-4f67-b9d3-793143654e9a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 11])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9ad8a9b8-e418-45b0-906f-2b7edd8ecea2, kind: Service { id: 31c86263-9349-4603-aaa0-6dc44827df57 }, name: Name("nexus-31c86263-9349-4603-aaa0-6dc44827df57"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 132, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9d824d0b-7228-40a5-a231-be10889211c7, kind: Service { id: 7c996f98-0c86-4480-8700-580fc6f3590a }, name: Name("nexus-7c996f98-0c86-4480-8700-580fc6f3590a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 254, 127])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
19122024-05-30T04:33:02.427ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19132024-05-30T04:33:02.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19142024-05-30T04:33:02.428ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19152024-05-30T04:33:02.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
19162024-05-30T04:33:02.430ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19172024-05-30T04:33:02.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19182024-05-30T04:33:02.432ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
19192024-05-30T04:33:02.432ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: e232750d-d52e-4ac4-8c87-d7b0e4145a1f, kind: Service { id: 8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3 }, name: Name("external-dns-8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 109])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: fb84270c-485b-4df1-a4a3-7bf184a5f27c, kind: Service { id: f72d0bad-b96e-4cb4-9157-126b67797b9b }, name: Name("external-dns-f72d0bad-b96e-4cb4-9157-126b67797b9b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }] }, VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1fbd52c8-c004-450c-ad40-e669ea7ac97a, kind: Service { id: b5bf7e90-33d3-4f67-b9d3-793143654e9a }, name: Name("nexus-b5bf7e90-33d3-4f67-b9d3-793143654e9a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 11])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9ad8a9b8-e418-45b0-906f-2b7edd8ecea2, kind: Service { id: 31c86263-9349-4603-aaa0-6dc44827df57 }, name: Name("nexus-31c86263-9349-4603-aaa0-6dc44827df57"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 132, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9d824d0b-7228-40a5-a231-be10889211c7, kind: Service { id: 7c996f98-0c86-4480-8700-580fc6f3590a }, name: Name("nexus-7c996f98-0c86-4480-8700-580fc6f3590a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 254, 127])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }] }]
19202024-05-30T04:33:02.432ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
19212024-05-30T04:33:02.456ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
19222024-05-30T04:33:02.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "dab47ddd-d2a7-4136-931b-b39e4db75da5", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19232024-05-30T04:33:02.457ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19242024-05-30T04:33:02.459ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1c6f2623-c60f-43dc-b95a-76000bb6b47b", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19252024-05-30T04:33:02.459ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19262024-05-30T04:33:02.460ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
19272024-05-30T04:33:02.460ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 320.67839ms iteration = 8
19282024-05-30T04:33:02.466ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
19292024-05-30T04:33:02.467ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "921e5da2-10cb-48e3-a7d2-130ec4e10275", "content-length": "691", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19302024-05-30T04:33:02.467ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
19312024-05-30T04:33:02.467ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
19322024-05-30T04:33:02.467ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
19332024-05-30T04:33:02.469ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 329.461181ms iteration = 11
19342024-05-30T04:33:02.476ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19352024-05-30T04:33:02.477ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b77acb32-b162-4371-873c-5597b58f8b00", "content-length": "1", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19362024-05-30T04:33:02.479ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
19372024-05-30T04:33:02.479ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19382024-05-30T04:33:02.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19392024-05-30T04:33:02.481ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 341.349528ms iteration = 11
19402024-05-30T04:33:02.502ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 362.42851ms iteration = 11
19412024-05-30T04:33:02.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
19422024-05-30T04:33:02.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "927e11b1-f142-48f8-8869-e033415b2451", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19432024-05-30T04:33:02.524ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19442024-05-30T04:33:02.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "991bd49c-3cc1-470d-951d-a03a31828ffb", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19452024-05-30T04:33:02.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19462024-05-30T04:33:02.528ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "304535f5-c72b-4636-a3da-1c6d9a02fc45", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19472024-05-30T04:33:02.528ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19482024-05-30T04:33:02.529ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
19492024-05-30T04:33:02.529ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 389.687163ms iteration = 8
19502024-05-30T04:33:02.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19512024-05-30T04:33:02.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19522024-05-30T04:33:02.561ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
19532024-05-30T04:33:02.561ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19542024-05-30T04:33:02.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19552024-05-30T04:33:02.568ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19562024-05-30T04:33:02.569ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "af414254-cf04-4b65-aa18-5fa434606f17", "content-length": "121", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19572024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
19582024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
19592024-05-30T04:33:02.569ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
19602024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
19612024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
19622024-05-30T04:33:02.569ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
19632024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
19642024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
19652024-05-30T04:33:02.569ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
19662024-05-30T04:33:02.569ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19672024-05-30T04:33:02.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19682024-05-30T04:33:02.571ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19692024-05-30T04:33:02.571ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0fb4ebc9-f1f3-4479-af66-85f723429fed", "content-length": "2", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19702024-05-30T04:33:02.571ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19712024-05-30T04:33:02.571ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a7278aa4-2678-45c8-8db4-353d03f31391", "content-length": "52", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19722024-05-30T04:33:02.571ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
19732024-05-30T04:33:02.571ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19742024-05-30T04:33:02.649ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3ba74ac6-33a3-4ad6-84be-83520b1e55fc", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19752024-05-30T04:33:02.649ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
19762024-05-30T04:33:02.649ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19772024-05-30T04:33:02.650ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bcc297f5-40c3-46e3-8844-d42725c8b1e9", "content-length": "467", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19782024-05-30T04:33:02.650ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19792024-05-30T04:33:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19802024-05-30T04:33:02.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19812024-05-30T04:33:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19822024-05-30T04:33:02.656ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19832024-05-30T04:33:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19842024-05-30T04:33:02.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19852024-05-30T04:33:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19862024-05-30T04:33:02.664ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 524.251702ms iteration = 11
19872024-05-30T04:33:02.689ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 73ed95a1-f279-488a-9386-6d3e4371db4d, time_created: 2024-05-30T04:28:21.534950Z, time_modified: 2024-05-30T04:28:21.534950Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: 0, usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
19882024-05-30T04:33:02.689ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
19892024-05-30T04:33:02.689ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19902024-05-30T04:33:02.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19912024-05-30T04:33:02.691ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19922024-05-30T04:33:02.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19932024-05-30T04:33:02.692ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19942024-05-30T04:33:02.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
19952024-05-30T04:33:02.725ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19962024-05-30T04:33:02.726ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ef44bfb3-1436-4baf-a6ad-13300129b5e8", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19972024-05-30T04:33:02.727ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:487
19982024-05-30T04:33:02.727ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 581.038906ms
19992024-05-30T04:33:02.727ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 581.050316ms iteration = 2
20002024-05-30T04:33:05.514ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:56567
20012024-05-30T04:33:05.514ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:56567 req_id = 51b882f3-c97e-448a-8661-bda6f1b58e7e roles = RoleSet { roles: {} } uri = /metrics/producers
20022024-05-30T04:33:05.514ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:56567 req_id = 51b882f3-c97e-448a-8661-bda6f1b58e7e resource = Database result = Ok(()) uri = /metrics/producers
20032024-05-30T04:33:05.546ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
20042024-05-30T04:33:05.546ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:56567 req_id = 51b882f3-c97e-448a-8661-bda6f1b58e7e roles = RoleSet { roles: {} } uri = /metrics/producers
20052024-05-30T04:33:05.547ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:56567 req_id = 51b882f3-c97e-448a-8661-bda6f1b58e7e resource = Database result = Ok(()) uri = /metrics/producers
20062024-05-30T04:33:05.575ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
20072024-05-30T04:33:05.576ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a88279da-6ae1-4f53-9434-3b9fa9f5960b", "date": "Thu, 30 May 2024 04:33:05 GMT"} })
20082024-05-30T04:33:05.576ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 432db3ab-1c9c-4d65-aed9-91b1c693bf73
20092024-05-30T04:33:05.578ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 64681 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:56567 req_id = 51b882f3-c97e-448a-8661-bda6f1b58e7e response_code = 201 uri = /metrics/producers
20102024-05-30T04:33:06.698ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:46100
20112024-05-30T04:33:06.698ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:46100 req_id = 8894981a-2473-400b-8259-20e6ec4a1691 roles = RoleSet { roles: {} } uri = /metrics/producers
20122024-05-30T04:33:06.698ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:46100 req_id = 8894981a-2473-400b-8259-20e6ec4a1691 resource = Database result = Ok(()) uri = /metrics/producers
20132024-05-30T04:33:06.729ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
20142024-05-30T04:33:06.730ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:46100 req_id = 8894981a-2473-400b-8259-20e6ec4a1691 roles = RoleSet { roles: {} } uri = /metrics/producers
20152024-05-30T04:33:06.730ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:46100 req_id = 8894981a-2473-400b-8259-20e6ec4a1691 resource = Database result = Ok(()) uri = /metrics/producers
20162024-05-30T04:33:06.762ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
20172024-05-30T04:33:06.762ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5f773ddb-647a-4042-bbd0-2447262d7053", "date": "Thu, 30 May 2024 04:33:06 GMT"} })
20182024-05-30T04:33:06.762ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 73ed95a1-f279-488a-9386-6d3e4371db4d
20192024-05-30T04:33:06.764ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 66085 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:46100 req_id = 8894981a-2473-400b-8259-20e6ec4a1691 response_code = 201 uri = /metrics/producers
20202024-05-30T04:33:08.719ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 503 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = c05a84c7-1625-41d9-a408-2f78e0ede547 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
20212024-05-30T04:33:12.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
20222024-05-30T04:33:12.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20232024-05-30T04:33:12.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20242024-05-30T04:33:12.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20252024-05-30T04:33:12.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20262024-05-30T04:33:12.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20272024-05-30T04:33:12.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20282024-05-30T04:33:12.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20292024-05-30T04:33:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20302024-05-30T04:33:12.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20312024-05-30T04:33:12.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
20322024-05-30T04:33:12.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20332024-05-30T04:33:12.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20342024-05-30T04:33:12.156ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
20352024-05-30T04:33:12.156ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.008476ms iteration = 32
20362024-05-30T04:33:18.719ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 805 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 41e51a88-dcfa-43c9-8658-90f6602e0704 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
20372024-05-30T04:33:22.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
20382024-05-30T04:33:22.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20392024-05-30T04:33:22.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20402024-05-30T04:33:22.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20412024-05-30T04:33:22.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20422024-05-30T04:33:22.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20432024-05-30T04:33:22.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20442024-05-30T04:33:22.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20452024-05-30T04:33:22.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20462024-05-30T04:33:22.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20472024-05-30T04:33:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
20482024-05-30T04:33:22.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20492024-05-30T04:33:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20502024-05-30T04:33:22.156ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
20512024-05-30T04:33:22.156ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.130317ms iteration = 33
20522024-05-30T04:33:28.720ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1111 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = a6b5a9f2-914e-4eeb-80a4-b2769ae5cee1 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
20532024-05-30T04:33:28.721ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
20542024-05-30T04:33:28.721ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
20552024-05-30T04:33:28.752ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
20562024-05-30T04:33:28.752ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:34308
20572024-05-30T04:33:28.752ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34308 req_id = a5ef6864-a5d2-4cf2-b688-79b8b89e7243 roles = RoleSet { roles: {} } uri = /metrics/producers
20582024-05-30T04:33:28.752ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34308 req_id = a5ef6864-a5d2-4cf2-b688-79b8b89e7243 resource = Database result = Ok(()) uri = /metrics/producers
20592024-05-30T04:33:28.783ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
20602024-05-30T04:33:28.784ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34308 req_id = a5ef6864-a5d2-4cf2-b688-79b8b89e7243 roles = RoleSet { roles: {} } uri = /metrics/producers
20612024-05-30T04:33:28.784ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34308 req_id = a5ef6864-a5d2-4cf2-b688-79b8b89e7243 resource = Database result = Ok(()) uri = /metrics/producers
20622024-05-30T04:33:28.789ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
20632024-05-30T04:33:28.789ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6c609471-0e83-4a95-a00f-e0606d42d22a", "date": "Thu, 30 May 2024 04:33:28 GMT"} })
20642024-05-30T04:33:28.789ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 31c86263-9349-4603-aaa0-6dc44827df57
20652024-05-30T04:33:28.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 38954 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:34308 req_id = a5ef6864-a5d2-4cf2-b688-79b8b89e7243 response_code = 201 uri = /metrics/producers
20662024-05-30T04:33:28.791ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "a5ef6864-a5d2-4cf2-b688-79b8b89e7243", "content-length": "41", "date": "Thu, 30 May 2024 04:33:28 GMT"} })
20672024-05-30T04:33:28.792ZDEBGnexus (producer-registration-task): registered with nexus successfully
20682024-05-30T04:33:28.792ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
20692024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
20702024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
20712024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20722024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20732024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
20742024-05-30T04:33:32.140ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
20752024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
20762024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
20772024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
20782024-05-30T04:33:32.140ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
20792024-05-30T04:33:32.140ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
20802024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20812024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.641\u{b5}s iteration = 12
20822024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20832024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
20842024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
20852024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20862024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20872024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
20882024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20892024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
20902024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20912024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20922024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20932024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20942024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20952024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20962024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20972024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20982024-05-30T04:33:32.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20992024-05-30T04:33:32.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21002024-05-30T04:33:32.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21012024-05-30T04:33:32.142ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
21022024-05-30T04:33:32.143ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.710381ms iteration = 12
21032024-05-30T04:33:32.143ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1234, tv_nsec: 254830238 } })
21042024-05-30T04:33:32.143ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
21052024-05-30T04:33:32.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21062024-05-30T04:33:32.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21072024-05-30T04:33:32.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21082024-05-30T04:33:32.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1234, tv_nsec: 256348226 } })
21092024-05-30T04:33:32.144ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
21102024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
21112024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21122024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21132024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21142024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21152024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21162024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21172024-05-30T04:33:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21182024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21192024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21202024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21212024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21222024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
21232024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21242024-05-30T04:33:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21252024-05-30T04:33:32.148ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1234, tv_nsec: 260050678 } })
21262024-05-30T04:33:32.148ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
21272024-05-30T04:33:32.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21282024-05-30T04:33:32.148ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
21292024-05-30T04:33:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21302024-05-30T04:33:32.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21312024-05-30T04:33:32.153ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21322024-05-30T04:33:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21332024-05-30T04:33:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21342024-05-30T04:33:32.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21352024-05-30T04:33:32.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21362024-05-30T04:33:32.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21372024-05-30T04:33:32.157ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21382024-05-30T04:33:32.157ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
21392024-05-30T04:33:32.157ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 17.510442ms iteration = 12
21402024-05-30T04:33:32.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21412024-05-30T04:33:32.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
21422024-05-30T04:33:32.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21432024-05-30T04:33:32.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21442024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21452024-05-30T04:33:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21462024-05-30T04:33:32.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21472024-05-30T04:33:32.167ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
21482024-05-30T04:33:32.167ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.283814ms iteration = 34
21492024-05-30T04:33:32.168ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21502024-05-30T04:33:32.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21512024-05-30T04:33:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21522024-05-30T04:33:32.169ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
21532024-05-30T04:33:32.169ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 29.254037ms iteration = 13
21542024-05-30T04:33:32.170ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21552024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21562024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21572024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21582024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21592024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21602024-05-30T04:33:32.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21612024-05-30T04:33:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21622024-05-30T04:33:32.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21632024-05-30T04:33:32.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
21642024-05-30T04:33:32.179ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21652024-05-30T04:33:32.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21662024-05-30T04:33:32.180ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 40.505756ms iteration = 12
21672024-05-30T04:33:32.233ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
21682024-05-30T04:33:32.233ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21692024-05-30T04:33:32.233ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e6e96a31-1c0e-4e6e-923a-62ff13624a91", "content-length": "26", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
21702024-05-30T04:33:32.233ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21712024-05-30T04:33:32.235ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21722024-05-30T04:33:32.235ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
21732024-05-30T04:33:32.235ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21742024-05-30T04:33:32.235ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dcf82d95-2530-4fb8-87e0-0f094c979a07", "content-length": "26", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
21752024-05-30T04:33:32.235ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21762024-05-30T04:33:32.237ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
21772024-05-30T04:33:32.238ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
21782024-05-30T04:33:32.238ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21792024-05-30T04:33:32.238ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "93662a14-0c23-4cd7-a23f-459c9427f5c7", "content-length": "26", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
21802024-05-30T04:33:32.238ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21812024-05-30T04:33:32.240ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21822024-05-30T04:33:32.282ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
21832024-05-30T04:33:32.282ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ce064ec2-c86b-40a0-b647-6ff3f3d2354c", "content-length": "691", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
21842024-05-30T04:33:32.282ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
21852024-05-30T04:33:32.282ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
21862024-05-30T04:33:32.282ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
21872024-05-30T04:33:32.283ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 143.48412ms iteration = 12
21882024-05-30T04:33:32.333ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
21892024-05-30T04:33:32.335ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "47cddc67-6463-4048-94c2-844889c95309", "content-length": "1", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
21902024-05-30T04:33:32.336ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21912024-05-30T04:33:32.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21922024-05-30T04:33:32.337ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 196.686683ms iteration = 12
21932024-05-30T04:33:32.338ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 197.750955ms iteration = 12
21942024-05-30T04:33:32.400ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21952024-05-30T04:33:32.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21962024-05-30T04:33:32.402ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
21972024-05-30T04:33:32.402ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21982024-05-30T04:33:32.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21992024-05-30T04:33:32.410ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
22002024-05-30T04:33:32.411ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7eb4c882-7405-4154-9996-3fb78b326ce1", "content-length": "121", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
22012024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
22022024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
22032024-05-30T04:33:32.411ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
22042024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
22052024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
22062024-05-30T04:33:32.411ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
22072024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
22082024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
22092024-05-30T04:33:32.411ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
22102024-05-30T04:33:32.411ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22112024-05-30T04:33:32.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22122024-05-30T04:33:32.413ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
22132024-05-30T04:33:32.413ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e316f407-1ffc-4550-b961-06915947df09", "content-length": "2", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
22142024-05-30T04:33:32.413ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
22152024-05-30T04:33:32.413ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a7482c0a-d8dc-4915-aba9-996acd0ad406", "content-length": "52", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
22162024-05-30T04:33:32.413ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
22172024-05-30T04:33:32.413ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
22182024-05-30T04:33:32.465ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9a0fafcd-e8ea-4ddf-b4cd-b93150323544", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
22192024-05-30T04:33:32.465ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
22202024-05-30T04:33:32.465ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
22212024-05-30T04:33:32.469ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "40fc6960-8ea5-450f-b287-2e3578964931", "content-length": "467", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
22222024-05-30T04:33:32.469ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22232024-05-30T04:33:32.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22242024-05-30T04:33:32.472ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22252024-05-30T04:33:32.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22262024-05-30T04:33:32.474ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22272024-05-30T04:33:32.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22282024-05-30T04:33:32.475ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22292024-05-30T04:33:32.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22302024-05-30T04:33:32.481ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 341.352498ms iteration = 12
22312024-05-30T04:33:38.791ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 775 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = bff25da5-0bad-459d-b93b-dc85b054bb42 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
22322024-05-30T04:33:42.139ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
22332024-05-30T04:33:42.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22342024-05-30T04:33:42.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22352024-05-30T04:33:42.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22362024-05-30T04:33:42.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22372024-05-30T04:33:42.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22382024-05-30T04:33:42.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22392024-05-30T04:33:42.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22402024-05-30T04:33:42.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22412024-05-30T04:33:42.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22422024-05-30T04:33:42.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
22432024-05-30T04:33:42.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22442024-05-30T04:33:42.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22452024-05-30T04:33:42.157ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
22462024-05-30T04:33:42.157ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.677464ms iteration = 35
22472024-05-30T04:33:48.791ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 932 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = a9942d3d-d265-4aaf-a6f8-559daa11c602 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
22482024-05-30T04:33:49.455ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:40922
22492024-05-30T04:33:49.455ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40922 req_id = adaf3e70-b0e0-4959-b2f2-db8a3f6231d0 roles = RoleSet { roles: {} } uri = /metrics/producers
22502024-05-30T04:33:49.455ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40922 req_id = adaf3e70-b0e0-4959-b2f2-db8a3f6231d0 resource = Database result = Ok(()) uri = /metrics/producers
22512024-05-30T04:33:49.487ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
22522024-05-30T04:33:49.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40922 req_id = adaf3e70-b0e0-4959-b2f2-db8a3f6231d0 roles = RoleSet { roles: {} } uri = /metrics/producers
22532024-05-30T04:33:49.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40922 req_id = adaf3e70-b0e0-4959-b2f2-db8a3f6231d0 resource = Database result = Ok(()) uri = /metrics/producers
22542024-05-30T04:33:49.519ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
22552024-05-30T04:33:49.520ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a3281b6d-2446-492b-9e99-44c42932bacf", "date": "Thu, 30 May 2024 04:33:49 GMT"} })
22562024-05-30T04:33:49.520ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 439e757e-803c-4f81-8f3d-8bd0f1f5d2cc
22572024-05-30T04:33:49.522ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 66808 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:40922 req_id = adaf3e70-b0e0-4959-b2f2-db8a3f6231d0 response_code = 201 uri = /metrics/producers
22582024-05-30T04:33:52.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
22592024-05-30T04:33:52.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22602024-05-30T04:33:52.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22612024-05-30T04:33:52.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22622024-05-30T04:33:52.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22632024-05-30T04:33:52.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22642024-05-30T04:33:52.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22652024-05-30T04:33:52.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22662024-05-30T04:33:52.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22672024-05-30T04:33:52.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22682024-05-30T04:33:52.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
22692024-05-30T04:33:52.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22702024-05-30T04:33:52.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22712024-05-30T04:33:52.156ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
22722024-05-30T04:33:52.156ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.242867ms iteration = 36
22732024-05-30T04:33:58.792ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1272 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 9215d77e-4743-44b9-80cc-e17e6fc7ab41 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
22742024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
22752024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
22762024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22772024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
22782024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22792024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
22802024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
22812024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
22822024-05-30T04:34:02.140ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
22832024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
22842024-05-30T04:34:02.140ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
22852024-05-30T04:34:02.140ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
22862024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
22872024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.12\u{b5}s iteration = 13
22882024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22892024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
22902024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
22912024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22922024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
22932024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22942024-05-30T04:34:02.140ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:24:02.140761224 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
22952024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
22962024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
22972024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
22982024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
22992024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23002024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
23012024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
23022024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23032024-05-30T04:34:02.140ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
23042024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
23052024-05-30T04:34:02.141ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
23062024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.46\u{b5}s iteration = 8
23072024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
23082024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
23092024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
23102024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23112024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23122024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
23132024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23142024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23152024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23162024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
23172024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23182024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23192024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23202024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23212024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23222024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23232024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23242024-05-30T04:34:02.141ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 252802745 } })
23252024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23262024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23272024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23282024-05-30T04:34:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23292024-05-30T04:34:02.142ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
23302024-05-30T04:34:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.75855ms iteration = 8
23312024-05-30T04:34:02.142ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 253959929 } })
23322024-05-30T04:34:02.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23332024-05-30T04:34:02.143ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
23342024-05-30T04:34:02.143ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.64598ms iteration = 8
23352024-05-30T04:34:02.144ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
23362024-05-30T04:34:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.413779ms iteration = 13
23372024-05-30T04:34:02.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 255498536 } })
23382024-05-30T04:34:02.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23392024-05-30T04:34:02.145ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
23402024-05-30T04:34:02.145ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 256364457 } })
23412024-05-30T04:34:02.145ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:24:02.140761224 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
23422024-05-30T04:34:02.145ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.40237ms iteration = 7
23432024-05-30T04:34:02.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23442024-05-30T04:34:02.145ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23452024-05-30T04:34:02.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23462024-05-30T04:34:02.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23472024-05-30T04:34:02.145ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
23482024-05-30T04:34:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
23492024-05-30T04:34:02.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23502024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23512024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
23522024-05-30T04:34:02.147ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
23532024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
23542024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23552024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
23562024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23572024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
23582024-05-30T04:34:02.147ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23592024-05-30T04:34:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23602024-05-30T04:34:02.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
23612024-05-30T04:34:02.148ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23622024-05-30T04:34:02.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23632024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23642024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23652024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23662024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23672024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23682024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23692024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23702024-05-30T04:34:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23712024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
23722024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23732024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23742024-05-30T04:34:02.150ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
23752024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23762024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
23772024-05-30T04:34:02.150ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
23782024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 3.51148ms iteration = 7
23792024-05-30T04:34:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23802024-05-30T04:34:02.150ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 262306875 } })
23812024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23822024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23832024-05-30T04:34:02.151ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
23842024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23852024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23862024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23872024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23882024-05-30T04:34:02.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23892024-05-30T04:34:02.152ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23902024-05-30T04:34:02.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23912024-05-30T04:34:02.153ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
23922024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.223941ms iteration = 13
23932024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23942024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
23952024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
23962024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
23972024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
23982024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23992024-05-30T04:34:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24002024-05-30T04:34:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24012024-05-30T04:34:02.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24022024-05-30T04:34:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24032024-05-30T04:34:02.155ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
24042024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
24052024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
24062024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24072024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24082024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24092024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
24102024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24112024-05-30T04:34:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
24122024-05-30T04:34:02.157ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24132024-05-30T04:34:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24142024-05-30T04:34:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24152024-05-30T04:34:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24162024-05-30T04:34:02.159ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24172024-05-30T04:34:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24182024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24192024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
24202024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24212024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24222024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24232024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24242024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24252024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24262024-05-30T04:34:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24272024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
24282024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
24292024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24302024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24312024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24322024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
24332024-05-30T04:34:02.163ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24342024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24352024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
24362024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
24372024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24382024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24392024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
24402024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.905835ms iteration = 37
24412024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24422024-05-30T04:34:02.165ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
24432024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 24.612163ms iteration = 14
24442024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24452024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24462024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24472024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24482024-05-30T04:34:02.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24492024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
24502024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24512024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24522024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24532024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24542024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24552024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24562024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24572024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24582024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
24592024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
24602024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
24612024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
24622024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
24632024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
24642024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 31.673504ms iteration = 8
24652024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 31.667614ms iteration = 8
24662024-05-30T04:34:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24672024-05-30T04:34:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24682024-05-30T04:34:02.177ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24692024-05-30T04:34:02.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24702024-05-30T04:34:02.178ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
24712024-05-30T04:34:02.178ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 37.868696ms iteration = 8
24722024-05-30T04:34:02.179ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24732024-05-30T04:34:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
24742024-05-30T04:34:02.182ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24752024-05-30T04:34:02.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24762024-05-30T04:34:02.184ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 43.718883ms iteration = 13
24772024-05-30T04:34:02.277ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
24782024-05-30T04:34:02.300ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
24792024-05-30T04:34:02.300ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24802024-05-30T04:34:02.300ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1af657ac-7cc0-4f57-9ea3-9308a523d7ee", "content-length": "26", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24812024-05-30T04:34:02.300ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24822024-05-30T04:34:02.302ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24832024-05-30T04:34:02.302ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
24842024-05-30T04:34:02.302ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24852024-05-30T04:34:02.303ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "62e25d3e-93a6-4a7f-9eb7-2df5cbf42f09", "content-length": "26", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24862024-05-30T04:34:02.303ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24872024-05-30T04:34:02.304ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
24882024-05-30T04:34:02.305ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
24892024-05-30T04:34:02.305ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24902024-05-30T04:34:02.305ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9d0136ff-3b25-4217-9b99-cf46f29ddbef", "content-length": "26", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24912024-05-30T04:34:02.305ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24922024-05-30T04:34:02.307ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
24932024-05-30T04:34:02.351ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
24942024-05-30T04:34:02.563ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
24952024-05-30T04:34:02.563ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9cd3df44-fd59-4824-ace9-5bef3d44aecf", "content-length": "691", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24962024-05-30T04:34:02.563ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
24972024-05-30T04:34:02.563ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
24982024-05-30T04:34:02.563ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
24992024-05-30T04:34:02.564ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 424.013568ms iteration = 13
25002024-05-30T04:34:02.610ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 469.494812ms iteration = 13
25012024-05-30T04:34:02.614ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
25022024-05-30T04:34:02.614ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "94dc8e97-de3e-41c9-bd65-31f8bf387f42", "content-length": "1", "date": "Thu, 30 May 2024 04:34:01 GMT"} })
25032024-05-30T04:34:02.616ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
25042024-05-30T04:34:02.616ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25052024-05-30T04:34:02.616ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cd038078-0bd1-47ca-bc83-d55a7cbe1385", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25062024-05-30T04:34:02.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25072024-05-30T04:34:02.616ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25082024-05-30T04:34:02.617ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 477.0991ms iteration = 13
25092024-05-30T04:34:02.618ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0e7826b5-5b45-48d1-a45d-116e4ccd0301", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25102024-05-30T04:34:02.618ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25112024-05-30T04:34:02.620ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
25122024-05-30T04:34:02.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 479.205774ms iteration = 9
25132024-05-30T04:34:02.622ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
25142024-05-30T04:34:02.659ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
25152024-05-30T04:34:02.659ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5c0962d8-de46-4bfc-9133-e8d30c599a39", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25162024-05-30T04:34:02.659ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25172024-05-30T04:34:02.661ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d4e043bf-9498-4059-86cd-8aaee7dfffc2", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25182024-05-30T04:34:02.661ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25192024-05-30T04:34:02.662ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "07083e4d-0606-4595-a31a-50b64ebe93a1", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25202024-05-30T04:34:02.662ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25212024-05-30T04:34:02.664ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
25222024-05-30T04:34:02.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 523.391122ms iteration = 9
25232024-05-30T04:34:02.688ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25242024-05-30T04:34:02.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25252024-05-30T04:34:02.690ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
25262024-05-30T04:34:02.690ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25272024-05-30T04:34:02.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25282024-05-30T04:34:02.697ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
25292024-05-30T04:34:02.697ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c1563397-56cd-4872-bef5-9ff06f34f944", "content-length": "121", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
25302024-05-30T04:34:02.697ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
25312024-05-30T04:34:02.697ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
25322024-05-30T04:34:02.698ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
25332024-05-30T04:34:02.698ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
25342024-05-30T04:34:02.698ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
25352024-05-30T04:34:02.698ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
25362024-05-30T04:34:02.698ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
25372024-05-30T04:34:02.698ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
25382024-05-30T04:34:02.698ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
25392024-05-30T04:34:02.698ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25402024-05-30T04:34:02.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25412024-05-30T04:34:02.699ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
25422024-05-30T04:34:02.700ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "beefb005-d6cf-4cdb-8b5a-ecd173b3e021", "content-length": "2", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
25432024-05-30T04:34:02.700ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
25442024-05-30T04:34:02.700ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8fce575b-c679-4277-bdbc-ce67e9c7873a", "content-length": "52", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
25452024-05-30T04:34:02.700ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
25462024-05-30T04:34:02.700ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
25472024-05-30T04:34:02.789ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "74d4b255-b7d0-4e24-9c16-c81e1d19e625", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
25482024-05-30T04:34:02.789ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
25492024-05-30T04:34:02.789ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
25502024-05-30T04:34:02.789ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9474b156-76d6-4e4b-bc5b-302e66da31d9", "content-length": "467", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
25512024-05-30T04:34:02.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25522024-05-30T04:34:02.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25532024-05-30T04:34:02.793ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25542024-05-30T04:34:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25552024-05-30T04:34:02.795ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25562024-05-30T04:34:02.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25572024-05-30T04:34:02.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25582024-05-30T04:34:02.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25592024-05-30T04:34:02.804ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 663.659456ms iteration = 13
25602024-05-30T04:34:08.792ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 777 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 24a894c8-ab4e-4cf1-ab71-532b1a52fdbb response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
25612024-05-30T04:34:12.140ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
25622024-05-30T04:34:12.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25632024-05-30T04:34:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25642024-05-30T04:34:12.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25652024-05-30T04:34:12.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25662024-05-30T04:34:12.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25672024-05-30T04:34:12.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25682024-05-30T04:34:12.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25692024-05-30T04:34:12.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25702024-05-30T04:34:12.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25712024-05-30T04:34:12.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
25722024-05-30T04:34:12.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25732024-05-30T04:34:12.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25742024-05-30T04:34:12.158ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
25752024-05-30T04:34:12.158ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.017218ms iteration = 38
25762024-05-30T04:34:18.791ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 437 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 2eabe307-a952-4b0b-9fce-680cd0357ee4 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
25772024-05-30T04:34:22.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
25782024-05-30T04:34:22.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25792024-05-30T04:34:22.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25802024-05-30T04:34:22.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25812024-05-30T04:34:22.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25822024-05-30T04:34:22.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25832024-05-30T04:34:22.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25842024-05-30T04:34:22.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25852024-05-30T04:34:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25862024-05-30T04:34:22.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25872024-05-30T04:34:22.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
25882024-05-30T04:34:22.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25892024-05-30T04:34:22.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25902024-05-30T04:34:22.157ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
25912024-05-30T04:34:22.157ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.177746ms iteration = 39
25922024-05-30T04:34:28.792ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 421 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 1591369b-1d3f-4251-b9ab-c01becb43443 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
25932024-05-30T04:34:29.308ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:33817
25942024-05-30T04:34:29.312ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33817 req_id = ee1a54da-426c-4d4c-826d-c73377693152 roles = RoleSet { roles: {} } uri = /probes/73ed95a1-f279-488a-9386-6d3e4371db4d
25952024-05-30T04:34:29.313ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33817 req_id = ee1a54da-426c-4d4c-826d-c73377693152 resource = Database result = Ok(()) uri = /probes/73ed95a1-f279-488a-9386-6d3e4371db4d
25962024-05-30T04:34:29.317ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 9159 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:33817 req_id = ee1a54da-426c-4d4c-826d-c73377693152 response_code = 200 uri = /probes/73ed95a1-f279-488a-9386-6d3e4371db4d
25972024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
25982024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25992024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
26002024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26012024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
26022024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
26032024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26042024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26052024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
26062024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26072024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
26082024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26092024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
26102024-05-30T04:34:32.141ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
26112024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
26122024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
26132024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26142024-05-30T04:34:32.141ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
26152024-05-30T04:34:32.141ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
26162024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
26172024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.39\u{b5}s iteration = 14
26182024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26192024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26202024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26212024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
26222024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26232024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26242024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26252024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26262024-05-30T04:34:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26272024-05-30T04:34:32.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26282024-05-30T04:34:32.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26292024-05-30T04:34:32.144ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
26302024-05-30T04:34:32.144ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.971174ms iteration = 14
26312024-05-30T04:34:32.144ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1294, tv_nsec: 255300984 } })
26322024-05-30T04:34:32.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26332024-05-30T04:34:32.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26342024-05-30T04:34:32.145ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
26352024-05-30T04:34:32.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26362024-05-30T04:34:32.145ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1294, tv_nsec: 256335226 } })
26372024-05-30T04:34:32.146ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
26382024-05-30T04:34:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26392024-05-30T04:34:32.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26402024-05-30T04:34:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26412024-05-30T04:34:32.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26422024-05-30T04:34:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26432024-05-30T04:34:32.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26442024-05-30T04:34:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26452024-05-30T04:34:32.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26462024-05-30T04:34:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26472024-05-30T04:34:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26482024-05-30T04:34:32.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26492024-05-30T04:34:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26502024-05-30T04:34:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26512024-05-30T04:34:32.149ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26522024-05-30T04:34:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26532024-05-30T04:34:32.149ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1294, tv_nsec: 260328882 } })
26542024-05-30T04:34:32.149ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
26552024-05-30T04:34:32.150ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
26562024-05-30T04:34:32.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26572024-05-30T04:34:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26582024-05-30T04:34:32.150ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26592024-05-30T04:34:32.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26602024-05-30T04:34:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26612024-05-30T04:34:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26622024-05-30T04:34:32.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26632024-05-30T04:34:32.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26642024-05-30T04:34:32.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26652024-05-30T04:34:32.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26662024-05-30T04:34:32.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26672024-05-30T04:34:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
26682024-05-30T04:34:32.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26692024-05-30T04:34:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26702024-05-30T04:34:32.159ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26712024-05-30T04:34:32.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26722024-05-30T04:34:32.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26732024-05-30T04:34:32.160ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
26742024-05-30T04:34:32.160ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.590985ms iteration = 40
26752024-05-30T04:34:32.161ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
26762024-05-30T04:34:32.161ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.948519ms iteration = 15
26772024-05-30T04:34:32.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26782024-05-30T04:34:32.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26792024-05-30T04:34:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26802024-05-30T04:34:32.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26812024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
26822024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26832024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
26842024-05-30T04:34:32.168ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26852024-05-30T04:34:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26862024-05-30T04:34:32.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26872024-05-30T04:34:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26882024-05-30T04:34:32.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26892024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
26902024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26912024-05-30T04:34:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26922024-05-30T04:34:32.175ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 34.458567ms iteration = 14
26932024-05-30T04:34:32.271ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
26942024-05-30T04:34:32.271ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
26952024-05-30T04:34:32.273ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "816562a0-128c-4bf0-a65f-d008a3a45556", "content-length": "26", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
26962024-05-30T04:34:32.273ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
26972024-05-30T04:34:32.274ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
26982024-05-30T04:34:32.275ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
26992024-05-30T04:34:32.275ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27002024-05-30T04:34:32.276ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ff68854-7b69-4bfa-b308-e5a0db3e4f9c", "content-length": "26", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
27012024-05-30T04:34:32.276ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27022024-05-30T04:34:32.277ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
27032024-05-30T04:34:32.289ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
27042024-05-30T04:34:32.289ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27052024-05-30T04:34:32.289ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e33d8454-4858-450c-85ff-ca181ab57852", "content-length": "26", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
27062024-05-30T04:34:32.289ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27072024-05-30T04:34:32.289ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/c3e31699-e123-4b80-8462-b2fb4fd4546f/state
27082024-05-30T04:34:32.290ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/c3e31699-e123-4b80-8462-b2fb4fd4546f/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4abe8fc5-0c76-4bf4-89c4-b1be8a9d208a", "content-length": "313", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
27092024-05-30T04:34:32.290ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f state = Running
27102024-05-30T04:34:32.290ZINFOnexus (BackgroundTasks): received new runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1982 instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f instance_state = InstanceRuntimeState { propolis_id: Some(5976ce1d-b479-45e7-940d-c7bd3ba7a7cb), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-05-30T04:34:21.668246734Z } propolis_id = 5976ce1d-b479-45e7-940d-c7bd3ba7a7cb vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-05-30T04:34:29.283435115Z }
27112024-05-30T04:34:32.290ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
27122024-05-30T04:34:32.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
27132024-05-30T04:34:32.290ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27142024-05-30T04:34:32.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
27152024-05-30T04:34:32.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
27162024-05-30T04:34:32.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
27172024-05-30T04:34:32.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
27182024-05-30T04:34:32.296ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
27192024-05-30T04:34:32.297ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0c52205c-c3fb-4ef6-9ee9-7c3a6bcdf2a4", "content-length": "811", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
27202024-05-30T04:34:32.297ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
27212024-05-30T04:34:32.297ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 241, 76, 112])), vni: Vni(14850883) }} sled = buskin
27222024-05-30T04:34:32.297ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
27232024-05-30T04:34:32.297ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
27242024-05-30T04:34:32.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
27252024-05-30T04:34:32.298ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
27262024-05-30T04:34:32.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
27272024-05-30T04:34:32.299ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 157.769845ms iteration = 14
27282024-05-30T04:34:32.299ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
27292024-05-30T04:34:32.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
27302024-05-30T04:34:32.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27312024-05-30T04:34:32.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, lookup_type: ById(d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab) }, key: 44b9d9f4-dd81-49cb-9277-bb9d9e2cb6e9, lookup_type: ById(44b9d9f4-dd81-49cb-9277-bb9d9e2cb6e9) }, key: c3e31699-e123-4b80-8462-b2fb4fd4546f, lookup_type: ById(c3e31699-e123-4b80-8462-b2fb4fd4546f) } result = Ok(())
27322024-05-30T04:34:32.305ZDEBGnexus (BackgroundTasks): instance state generation already advanced, won't touch network config background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f
27332024-05-30T04:34:32.308ZINFOnexus (BackgroundTasks): instance and vmm updated by sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2116 instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f instance_updated = false propolis_id = 5976ce1d-b479-45e7-940d-c7bd3ba7a7cb vmm_updated = false
27342024-05-30T04:34:32.308ZDEBGnexus (BackgroundTasks): update successful background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f instance_updated = false state = Running vmm_updated = false
27352024-05-30T04:34:32.310ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1
27362024-05-30T04:34:32.310ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 169.300738ms iteration = 14
27372024-05-30T04:34:32.369ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
27382024-05-30T04:34:32.369ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2c2a19f4-7ec6-49d6-a0c7-e992a6c1d5cd", "content-length": "2", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
27392024-05-30T04:34:32.370ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27402024-05-30T04:34:32.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27412024-05-30T04:34:32.372ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 231.25205ms iteration = 14
27422024-05-30T04:34:32.377ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 236.438781ms iteration = 14
27432024-05-30T04:34:32.439ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27442024-05-30T04:34:32.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27452024-05-30T04:34:32.442ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
27462024-05-30T04:34:32.442ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27472024-05-30T04:34:32.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27482024-05-30T04:34:32.448ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
27492024-05-30T04:34:32.449ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "606cf8a5-1491-47f5-b77d-337a6eea1cfd", "content-length": "121", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
27502024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
27512024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
27522024-05-30T04:34:32.449ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
27532024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
27542024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
27552024-05-30T04:34:32.449ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
27562024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
27572024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
27582024-05-30T04:34:32.449ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
27592024-05-30T04:34:32.449ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27602024-05-30T04:34:32.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27612024-05-30T04:34:32.451ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
27622024-05-30T04:34:32.451ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9987accd-b16c-4b69-ae33-044bc9f6d571", "content-length": "2", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
27632024-05-30T04:34:32.451ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
27642024-05-30T04:34:32.452ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "51c7407d-a64d-4342-a8a5-34e3a61c9f12", "content-length": "52", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
27652024-05-30T04:34:32.452ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
27662024-05-30T04:34:32.452ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27672024-05-30T04:34:32.503ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "039fe199-cba9-490e-aa06-07b3f0a71eae", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
27682024-05-30T04:34:32.503ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
27692024-05-30T04:34:32.503ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27702024-05-30T04:34:32.503ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "527f1059-5c4e-4997-bbbe-38100feeafe5", "content-length": "467", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
27712024-05-30T04:34:32.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27722024-05-30T04:34:32.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27732024-05-30T04:34:32.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27742024-05-30T04:34:32.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27752024-05-30T04:34:32.509ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27762024-05-30T04:34:32.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27772024-05-30T04:34:32.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27782024-05-30T04:34:32.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27792024-05-30T04:34:32.517ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 375.928425ms iteration = 14
27802024-05-30T04:34:38.794ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 2546 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = cf640340-c396-4517-a6d7-6b1aeb9eae34 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
27812024-05-30T04:34:39.029ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:65103
27822024-05-30T04:34:39.029ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65103 req_id = 17e5c7d0-a945-47d0-9846-ea301fc15f5a roles = RoleSet { roles: {} } uri = /metrics/producers
27832024-05-30T04:34:39.030ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65103 req_id = 17e5c7d0-a945-47d0-9846-ea301fc15f5a resource = Database result = Ok(()) uri = /metrics/producers
27842024-05-30T04:34:39.061ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
27852024-05-30T04:34:39.061ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65103 req_id = 17e5c7d0-a945-47d0-9846-ea301fc15f5a roles = RoleSet { roles: {} } uri = /metrics/producers
27862024-05-30T04:34:39.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65103 req_id = 17e5c7d0-a945-47d0-9846-ea301fc15f5a resource = Database result = Ok(()) uri = /metrics/producers
27872024-05-30T04:34:39.066ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
27882024-05-30T04:34:39.066ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "93bff06a-8de2-42f3-8596-281f2cbeb7d5", "date": "Thu, 30 May 2024 04:34:39 GMT"} })
27892024-05-30T04:34:39.066ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 0cf65393-f6a2-480b-8f0b-78c9543ba992
27902024-05-30T04:34:39.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 38367 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:65103 req_id = 17e5c7d0-a945-47d0-9846-ea301fc15f5a response_code = 201 uri = /metrics/producers
27912024-05-30T04:34:42.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
27922024-05-30T04:34:42.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27932024-05-30T04:34:42.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27942024-05-30T04:34:42.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27952024-05-30T04:34:42.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27962024-05-30T04:34:42.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27972024-05-30T04:34:42.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27982024-05-30T04:34:42.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27992024-05-30T04:34:42.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28002024-05-30T04:34:42.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28012024-05-30T04:34:42.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
28022024-05-30T04:34:42.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28032024-05-30T04:34:42.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28042024-05-30T04:34:42.157ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
28052024-05-30T04:34:42.157ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.336908ms iteration = 41
28062024-05-30T04:34:48.793ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 550 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 0f55f4a2-d4ad-4aa5-96f5-8dc525410887 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
28072024-05-30T04:34:52.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
28082024-05-30T04:34:52.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28092024-05-30T04:34:52.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28102024-05-30T04:34:52.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28112024-05-30T04:34:52.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28122024-05-30T04:34:52.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28132024-05-30T04:34:52.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28142024-05-30T04:34:52.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28152024-05-30T04:34:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28162024-05-30T04:34:52.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28172024-05-30T04:34:52.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
28182024-05-30T04:34:52.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28192024-05-30T04:34:52.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28202024-05-30T04:34:52.158ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
28212024-05-30T04:34:52.158ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.821033ms iteration = 42
28222024-05-30T04:34:58.792ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 860 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 6592018a-815b-4490-a46b-21f904e42742 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
28232024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
28242024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28252024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
28262024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
28272024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
28282024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28292024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
28302024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28312024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28322024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
28332024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
28342024-05-30T04:35:02.141ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
28352024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
28362024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
28372024-05-30T04:35:02.141ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
28382024-05-30T04:35:02.141ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
28392024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
28402024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.59\u{b5}s iteration = 15
28412024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
28422024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28432024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
28442024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28452024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
28462024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28472024-05-30T04:35:02.141ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:25:02.141226928 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
28482024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
28492024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
28502024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
28512024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
28522024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28532024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
28542024-05-30T04:35:02.141ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
28552024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.61\u{b5}s iteration = 9
28562024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
28572024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
28582024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
28592024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28602024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
28612024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
28622024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28632024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28642024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
28652024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28662024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28672024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28682024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28692024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28702024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
28712024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28722024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28732024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28742024-05-30T04:35:02.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28752024-05-30T04:35:02.142ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 252407251 } })
28762024-05-30T04:35:02.142ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28772024-05-30T04:35:02.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28782024-05-30T04:35:02.142ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 252954947 } })
28792024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28802024-05-30T04:35:02.144ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:25:02.141226928 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
28812024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.958795ms iteration = 8
28822024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28832024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
28842024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
28852024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.956283ms iteration = 9
28862024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.013385ms iteration = 9
28872024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28882024-05-30T04:35:02.144ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
28892024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.376639ms iteration = 15
28902024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28912024-05-30T04:35:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28922024-05-30T04:35:02.145ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 255809060 } })
28932024-05-30T04:35:02.145ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
28942024-05-30T04:35:02.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28952024-05-30T04:35:02.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28962024-05-30T04:35:02.146ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 256847992 } })
28972024-05-30T04:35:02.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28982024-05-30T04:35:02.146ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
28992024-05-30T04:35:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
29002024-05-30T04:35:02.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29012024-05-30T04:35:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29022024-05-30T04:35:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29032024-05-30T04:35:02.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29042024-05-30T04:35:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29052024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
29062024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
29072024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
29082024-05-30T04:35:02.148ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
29092024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
29102024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
29112024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29122024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
29132024-05-30T04:35:02.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29142024-05-30T04:35:02.149ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
29152024-05-30T04:35:02.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29162024-05-30T04:35:02.149ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 259436062 } })
29172024-05-30T04:35:02.149ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
29182024-05-30T04:35:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29192024-05-30T04:35:02.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29202024-05-30T04:35:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29212024-05-30T04:35:02.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29222024-05-30T04:35:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29232024-05-30T04:35:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29242024-05-30T04:35:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29252024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29262024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29272024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
29282024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
29292024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
29302024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29312024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
29322024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
29332024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
29342024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29352024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29362024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29372024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29382024-05-30T04:35:02.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29392024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29402024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
29412024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29422024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29432024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29442024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29452024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29462024-05-30T04:35:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29472024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29482024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29492024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29502024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29512024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29522024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29532024-05-30T04:35:02.156ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29542024-05-30T04:35:02.157ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
29552024-05-30T04:35:02.157ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 16.409418ms iteration = 15
29562024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
29572024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29582024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29592024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29602024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29612024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29622024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29632024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
29642024-05-30T04:35:02.159ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
29652024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 10.951246ms iteration = 8
29662024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29672024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29682024-05-30T04:35:02.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29692024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29702024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
29712024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29722024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29732024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
29742024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
29752024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29762024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29772024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
29782024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
29792024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29802024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29812024-05-30T04:35:02.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29822024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29832024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29842024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29852024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29862024-05-30T04:35:02.164ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
29872024-05-30T04:35:02.164ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 22.855013ms iteration = 16
29882024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29892024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29902024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29912024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29922024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
29932024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.047929ms iteration = 43
29942024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29952024-05-30T04:35:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29962024-05-30T04:35:02.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29972024-05-30T04:35:02.168ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29982024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29992024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30002024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30012024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30022024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30032024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30042024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
30052024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
30062024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
30072024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 29.261587ms iteration = 9
30082024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30092024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30102024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30112024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30122024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30132024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
30142024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
30152024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
30162024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 31.687735ms iteration = 9
30172024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30182024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30192024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30202024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
30212024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30222024-05-30T04:35:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30232024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 40.349815ms iteration = 15
30242024-05-30T04:35:02.223ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
30252024-05-30T04:35:02.223ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 81.752571ms iteration = 9
30262024-05-30T04:35:02.284ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
30272024-05-30T04:35:02.296ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
30282024-05-30T04:35:02.296ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30292024-05-30T04:35:02.296ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5520fe15-39d4-4e1f-99c5-fd328887e34f", "content-length": "26", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
30302024-05-30T04:35:02.296ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30312024-05-30T04:35:02.297ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30322024-05-30T04:35:02.307ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
30332024-05-30T04:35:02.307ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30342024-05-30T04:35:02.308ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1074dbbe-69cf-451d-bfac-ec522b7d4673", "content-length": "26", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
30352024-05-30T04:35:02.308ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30362024-05-30T04:35:02.311ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
30372024-05-30T04:35:02.335ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
30382024-05-30T04:35:02.375ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30392024-05-30T04:35:02.375ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b4494d3b-56b2-49ba-90ef-b8a89e144325", "content-length": "691", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
30402024-05-30T04:35:02.375ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
30412024-05-30T04:35:02.375ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
30422024-05-30T04:35:02.375ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
30432024-05-30T04:35:02.378ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 237.057468ms iteration = 15
30442024-05-30T04:35:02.380ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
30452024-05-30T04:35:02.380ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30462024-05-30T04:35:02.380ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ef34c5-44e2-41cc-93a0-efcd9148d80d", "content-length": "26", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
30472024-05-30T04:35:02.380ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30482024-05-30T04:35:02.381ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30492024-05-30T04:35:02.429ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
30502024-05-30T04:35:02.488ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 347.60808ms iteration = 15
30512024-05-30T04:35:02.501ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
30522024-05-30T04:35:02.501ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ea496a9e-9b21-46b6-b90c-5d41b35d1709", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30532024-05-30T04:35:02.501ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30542024-05-30T04:35:02.502ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "aeaa43eb-1712-4b9b-8ade-df4f1f366301", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30552024-05-30T04:35:02.502ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30562024-05-30T04:35:02.503ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30572024-05-30T04:35:02.503ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 362.196107ms iteration = 10
30582024-05-30T04:35:02.553ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
30592024-05-30T04:35:02.553ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "85f60f36-de23-4c3f-99cf-dba153a8d75b", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30602024-05-30T04:35:02.553ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30612024-05-30T04:35:02.555ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "08002a64-1058-43b9-854b-35ddaf8da1e7", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30622024-05-30T04:35:02.555ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30632024-05-30T04:35:02.557ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f910ed69-6b82-42f4-bfb6-f7fa4b520c5d", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30642024-05-30T04:35:02.557ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30652024-05-30T04:35:02.559ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
30662024-05-30T04:35:02.559ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 418.609776ms iteration = 10
30672024-05-30T04:35:02.605ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
30682024-05-30T04:35:02.606ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f8be4cbd-8575-4a00-bd9f-a036fa9c0e7f", "content-length": "2", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
30692024-05-30T04:35:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30702024-05-30T04:35:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30712024-05-30T04:35:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 468.039935ms iteration = 15
30722024-05-30T04:35:02.652ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30732024-05-30T04:35:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30742024-05-30T04:35:02.654ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
30752024-05-30T04:35:02.654ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30762024-05-30T04:35:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30772024-05-30T04:35:02.662ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
30782024-05-30T04:35:02.662ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "11649899-dab9-4609-a2b0-1f791780e535", "content-length": "121", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
30792024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
30802024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
30812024-05-30T04:35:02.662ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
30822024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
30832024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
30842024-05-30T04:35:02.662ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
30852024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
30862024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
30872024-05-30T04:35:02.662ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
30882024-05-30T04:35:02.662ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30892024-05-30T04:35:02.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30902024-05-30T04:35:02.664ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
30912024-05-30T04:35:02.664ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1d84e24d-ae35-44f1-8376-725f1764a695", "content-length": "2", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
30922024-05-30T04:35:02.664ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
30932024-05-30T04:35:02.665ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "03710756-3ae9-4719-a610-7a58fde7da72", "content-length": "52", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
30942024-05-30T04:35:02.665ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
30952024-05-30T04:35:02.665ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
30962024-05-30T04:35:02.710ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "aed80bdd-c514-4b5b-9972-638cc36de926", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
30972024-05-30T04:35:02.710ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
30982024-05-30T04:35:02.710ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
30992024-05-30T04:35:02.711ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a1181f48-9304-4212-8b9a-0213c72ae1e2", "content-length": "467", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
31002024-05-30T04:35:02.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31012024-05-30T04:35:02.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31022024-05-30T04:35:02.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31032024-05-30T04:35:02.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31042024-05-30T04:35:02.717ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31052024-05-30T04:35:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31062024-05-30T04:35:02.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31072024-05-30T04:35:02.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31082024-05-30T04:35:02.725ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 584.05782ms iteration = 15
31092024-05-30T04:35:08.793ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 534 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 2908b5af-b66a-424a-95ce-24a0217c1738 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
31102024-05-30T04:35:12.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
31112024-05-30T04:35:12.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31122024-05-30T04:35:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31132024-05-30T04:35:12.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31142024-05-30T04:35:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31152024-05-30T04:35:12.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31162024-05-30T04:35:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31172024-05-30T04:35:12.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31182024-05-30T04:35:12.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31192024-05-30T04:35:12.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31202024-05-30T04:35:12.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
31212024-05-30T04:35:12.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31222024-05-30T04:35:12.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31232024-05-30T04:35:12.157ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
31242024-05-30T04:35:12.157ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.039895ms iteration = 44
31252024-05-30T04:35:18.792ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 569 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = f89f7a6e-f569-4a30-8d66-415bb1164f46 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
31262024-05-30T04:35:22.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
31272024-05-30T04:35:22.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31282024-05-30T04:35:22.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31292024-05-30T04:35:22.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31302024-05-30T04:35:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31312024-05-30T04:35:22.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31322024-05-30T04:35:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31332024-05-30T04:35:22.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31342024-05-30T04:35:22.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31352024-05-30T04:35:22.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31362024-05-30T04:35:22.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
31372024-05-30T04:35:22.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31382024-05-30T04:35:22.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31392024-05-30T04:35:22.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
31402024-05-30T04:35:22.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.34384ms iteration = 45
31412024-05-30T04:35:28.793ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 803 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 41b9a572-51d7-443c-92dc-37d4b743c15d response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
31422024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
31432024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
31442024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31452024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
31462024-05-30T04:35:32.142ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
31472024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31482024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
31492024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
31502024-05-30T04:35:32.142ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
31512024-05-30T04:35:32.142ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
31522024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 11.45\u{b5}s iteration = 16
31532024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
31542024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31552024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
31562024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
31572024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
31582024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31592024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
31602024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31612024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31622024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
31632024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31642024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31652024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31662024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31672024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
31682024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31692024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31702024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31712024-05-30T04:35:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31722024-05-30T04:35:32.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31732024-05-30T04:35:32.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31742024-05-30T04:35:32.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31752024-05-30T04:35:32.145ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1354, tv_nsec: 255048681 } })
31762024-05-30T04:35:32.145ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31772024-05-30T04:35:32.145ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
31782024-05-30T04:35:32.145ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.456809ms iteration = 16
31792024-05-30T04:35:32.145ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
31802024-05-30T04:35:32.145ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31812024-05-30T04:35:32.145ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1354, tv_nsec: 255738949 } })
31822024-05-30T04:35:32.146ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
31832024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
31842024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31852024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31862024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
31872024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31882024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
31892024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
31902024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31912024-05-30T04:35:32.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
31922024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31932024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31942024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31952024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31962024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31972024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31982024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31992024-05-30T04:35:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32002024-05-30T04:35:32.150ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1354, tv_nsec: 260483044 } })
32012024-05-30T04:35:32.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32022024-05-30T04:35:32.150ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
32032024-05-30T04:35:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32042024-05-30T04:35:32.151ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
32052024-05-30T04:35:32.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32062024-05-30T04:35:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32072024-05-30T04:35:32.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32082024-05-30T04:35:32.152ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32092024-05-30T04:35:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32102024-05-30T04:35:32.154ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32112024-05-30T04:35:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
32122024-05-30T04:35:32.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32132024-05-30T04:35:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32142024-05-30T04:35:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32152024-05-30T04:35:32.155ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
32162024-05-30T04:35:32.155ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.158901ms iteration = 16
32172024-05-30T04:35:32.158ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32182024-05-30T04:35:32.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32192024-05-30T04:35:32.160ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32202024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
32212024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.983338ms iteration = 46
32222024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32232024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32242024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32252024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32262024-05-30T04:35:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32272024-05-30T04:35:32.162ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
32282024-05-30T04:35:32.162ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.515856ms iteration = 17
32292024-05-30T04:35:32.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32302024-05-30T04:35:32.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32312024-05-30T04:35:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32322024-05-30T04:35:32.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32332024-05-30T04:35:32.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32342024-05-30T04:35:32.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32352024-05-30T04:35:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32362024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
32372024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32382024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32392024-05-30T04:35:32.171ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.945074ms iteration = 16
32402024-05-30T04:35:32.256ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
32412024-05-30T04:35:32.256ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32422024-05-30T04:35:32.257ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "efd2f5b0-cb40-4ad2-9e2f-d5e2423f7048", "content-length": "26", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
32432024-05-30T04:35:32.257ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32442024-05-30T04:35:32.258ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
32452024-05-30T04:35:32.258ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32462024-05-30T04:35:32.259ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85ed208f-86e9-43ec-aae9-c0b4be755ca2", "content-length": "26", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
32472024-05-30T04:35:32.259ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32482024-05-30T04:35:32.259ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32492024-05-30T04:35:32.260ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32502024-05-30T04:35:32.365ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 223.193298ms iteration = 16
32512024-05-30T04:35:32.366ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
32522024-05-30T04:35:32.366ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e6ea0837-28b7-48cd-915b-090219b23946", "content-length": "2", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
32532024-05-30T04:35:32.366ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32542024-05-30T04:35:32.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32552024-05-30T04:35:32.368ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 226.740789ms iteration = 16
32562024-05-30T04:35:32.378ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32572024-05-30T04:35:32.378ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0343a5f8-aebf-420c-ac4d-2fe4b52482b6", "content-length": "691", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
32582024-05-30T04:35:32.379ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
32592024-05-30T04:35:32.379ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
32602024-05-30T04:35:32.379ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
32612024-05-30T04:35:32.379ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
32622024-05-30T04:35:32.379ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32632024-05-30T04:35:32.379ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "21199e46-a1e0-4d78-bfa5-a0bbd0790834", "content-length": "26", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
32642024-05-30T04:35:32.379ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32652024-05-30T04:35:32.379ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 237.741625ms iteration = 16
32662024-05-30T04:35:32.381ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
32672024-05-30T04:35:32.469ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32682024-05-30T04:35:32.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32692024-05-30T04:35:32.471ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
32702024-05-30T04:35:32.471ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32712024-05-30T04:35:32.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32722024-05-30T04:35:32.479ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
32732024-05-30T04:35:32.479ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3baa0418-a93a-42de-85f8-86ae327fd81a", "content-length": "121", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
32742024-05-30T04:35:32.479ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
32752024-05-30T04:35:32.480ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
32762024-05-30T04:35:32.480ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
32772024-05-30T04:35:32.480ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
32782024-05-30T04:35:32.480ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
32792024-05-30T04:35:32.480ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
32802024-05-30T04:35:32.480ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
32812024-05-30T04:35:32.480ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
32822024-05-30T04:35:32.480ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
32832024-05-30T04:35:32.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32842024-05-30T04:35:32.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32852024-05-30T04:35:32.481ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
32862024-05-30T04:35:32.482ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6bb34552-7fd2-45f5-9607-4e48cf001347", "content-length": "2", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
32872024-05-30T04:35:32.482ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
32882024-05-30T04:35:32.482ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "50138210-15c1-481d-9e16-bc3b44f929a2", "content-length": "52", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
32892024-05-30T04:35:32.482ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
32902024-05-30T04:35:32.482ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32912024-05-30T04:35:32.536ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "13bd31be-c861-4e50-b619-79179884ee73", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
32922024-05-30T04:35:32.536ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
32932024-05-30T04:35:32.536ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
32942024-05-30T04:35:32.537ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7ab7308d-846b-4ad3-b705-0327c8022c12", "content-length": "467", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
32952024-05-30T04:35:32.537ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32962024-05-30T04:35:32.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32972024-05-30T04:35:32.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32982024-05-30T04:35:32.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32992024-05-30T04:35:32.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33002024-05-30T04:35:32.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33012024-05-30T04:35:32.544ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33022024-05-30T04:35:32.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33032024-05-30T04:35:32.548ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 406.361246ms iteration = 16
33042024-05-30T04:35:35.617ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:33378
33052024-05-30T04:35:35.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33378 req_id = 809b7394-dfcd-4fd4-979d-6ba915429908 roles = RoleSet { roles: {} } uri = /metrics/producers
33062024-05-30T04:35:35.618ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33378 req_id = 809b7394-dfcd-4fd4-979d-6ba915429908 resource = Database result = Ok(()) uri = /metrics/producers
33072024-05-30T04:35:35.650ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
33082024-05-30T04:35:35.650ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33378 req_id = 809b7394-dfcd-4fd4-979d-6ba915429908 roles = RoleSet { roles: {} } uri = /metrics/producers
33092024-05-30T04:35:35.650ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33378 req_id = 809b7394-dfcd-4fd4-979d-6ba915429908 resource = Database result = Ok(()) uri = /metrics/producers
33102024-05-30T04:35:35.657ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
33112024-05-30T04:35:35.658ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9dccd58f-0b7f-4921-813e-43c450a06f2d", "date": "Thu, 30 May 2024 04:35:35 GMT"} })
33122024-05-30T04:35:35.658ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 432db3ab-1c9c-4d65-aed9-91b1c693bf73
33132024-05-30T04:35:35.660ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 43076 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33378 req_id = 809b7394-dfcd-4fd4-979d-6ba915429908 response_code = 201 uri = /metrics/producers
33142024-05-30T04:35:36.804ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1]:48577
33152024-05-30T04:35:36.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:48577 req_id = f2b47e62-1e84-495f-a4ba-bf7eb837b1a9 roles = RoleSet { roles: {} } uri = /metrics/producers
33162024-05-30T04:35:36.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:48577 req_id = f2b47e62-1e84-495f-a4ba-bf7eb837b1a9 resource = Database result = Ok(()) uri = /metrics/producers
33172024-05-30T04:35:36.839ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
33182024-05-30T04:35:36.840ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:48577 req_id = f2b47e62-1e84-495f-a4ba-bf7eb837b1a9 roles = RoleSet { roles: {} } uri = /metrics/producers
33192024-05-30T04:35:36.840ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:48577 req_id = f2b47e62-1e84-495f-a4ba-bf7eb837b1a9 resource = Database result = Ok(()) uri = /metrics/producers
33202024-05-30T04:35:36.847ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
33212024-05-30T04:35:36.847ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a7d7fbeb-da03-4fdb-a271-1ed6fe267959", "date": "Thu, 30 May 2024 04:35:36 GMT"} })
33222024-05-30T04:35:36.847ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 73ed95a1-f279-488a-9386-6d3e4371db4d
33232024-05-30T04:35:36.848ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 44042 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:48577 req_id = f2b47e62-1e84-495f-a4ba-bf7eb837b1a9 response_code = 201 uri = /metrics/producers
33242024-05-30T04:35:38.793ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 796 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 90b19d50-75c5-441f-8502-d795dc76a713 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
33252024-05-30T04:35:42.141ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
33262024-05-30T04:35:42.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33272024-05-30T04:35:42.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33282024-05-30T04:35:42.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33292024-05-30T04:35:42.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33302024-05-30T04:35:42.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33312024-05-30T04:35:42.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33322024-05-30T04:35:42.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33332024-05-30T04:35:42.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33342024-05-30T04:35:42.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33352024-05-30T04:35:42.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
33362024-05-30T04:35:42.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33372024-05-30T04:35:42.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33382024-05-30T04:35:42.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
33392024-05-30T04:35:42.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.682104ms iteration = 47
33402024-05-30T04:35:48.794ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1142 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 4e6714c8-6c53-4525-9f64-129e64b831dc response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
33412024-05-30T04:35:52.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
33422024-05-30T04:35:52.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33432024-05-30T04:35:52.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33442024-05-30T04:35:52.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33452024-05-30T04:35:52.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33462024-05-30T04:35:52.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33472024-05-30T04:35:52.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33482024-05-30T04:35:52.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33492024-05-30T04:35:52.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33502024-05-30T04:35:52.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33512024-05-30T04:35:52.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
33522024-05-30T04:35:52.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33532024-05-30T04:35:52.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33542024-05-30T04:35:52.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
33552024-05-30T04:35:52.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.415969ms iteration = 48
33562024-05-30T04:35:58.794ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 747 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 15960858-3946-4535-87f5-1d441237aa58 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
33572024-05-30T04:35:58.795ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
33582024-05-30T04:35:58.795ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
33592024-05-30T04:35:58.824ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
33602024-05-30T04:35:58.824ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:63343
33612024-05-30T04:35:58.824ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63343 req_id = 7c5810d2-ad4d-40dc-aac3-5dfa2a04ace7 roles = RoleSet { roles: {} } uri = /metrics/producers
33622024-05-30T04:35:58.825ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63343 req_id = 7c5810d2-ad4d-40dc-aac3-5dfa2a04ace7 resource = Database result = Ok(()) uri = /metrics/producers
33632024-05-30T04:35:58.857ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
33642024-05-30T04:35:58.857ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63343 req_id = 7c5810d2-ad4d-40dc-aac3-5dfa2a04ace7 roles = RoleSet { roles: {} } uri = /metrics/producers
33652024-05-30T04:35:58.857ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63343 req_id = 7c5810d2-ad4d-40dc-aac3-5dfa2a04ace7 resource = Database result = Ok(()) uri = /metrics/producers
33662024-05-30T04:35:58.862ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
33672024-05-30T04:35:58.862ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "358446d4-9e55-4067-836c-677e23942c32", "date": "Thu, 30 May 2024 04:35:58 GMT"} })
33682024-05-30T04:35:58.862ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 31c86263-9349-4603-aaa0-6dc44827df57
33692024-05-30T04:35:58.865ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 40315 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:63343 req_id = 7c5810d2-ad4d-40dc-aac3-5dfa2a04ace7 response_code = 201 uri = /metrics/producers
33702024-05-30T04:35:58.865ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "7c5810d2-ad4d-40dc-aac3-5dfa2a04ace7", "content-length": "41", "date": "Thu, 30 May 2024 04:35:58 GMT"} })
33712024-05-30T04:35:58.866ZDEBGnexus (producer-registration-task): registered with nexus successfully
33722024-05-30T04:35:58.866ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
33732024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
33742024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
33752024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33762024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
33772024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
33782024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
33792024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
33802024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
33812024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
33822024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
33832024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
33842024-05-30T04:36:02.142ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:26:02.142709894 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
33852024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
33862024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
33872024-05-30T04:36:02.142ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
33882024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 19.82\u{b5}s iteration = 10
33892024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
33902024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
33912024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
33922024-05-30T04:36:02.142ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
33932024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
33942024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33952024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
33962024-05-30T04:36:02.142ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
33972024-05-30T04:36:02.142ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
33982024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
33992024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34002024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7\u{b5}s iteration = 17
34012024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
34022024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
34032024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
34042024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34052024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
34062024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34072024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
34082024-05-30T04:36:02.142ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
34092024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
34102024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34112024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
34122024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34132024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34142024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34152024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
34162024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34172024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34182024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34192024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
34202024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34212024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
34222024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34232024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34242024-05-30T04:36:02.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34252024-05-30T04:36:02.144ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1384, tv_nsec: 253531274 } })
34262024-05-30T04:36:02.144ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1384, tv_nsec: 253536884 } })
34272024-05-30T04:36:02.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34282024-05-30T04:36:02.145ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34292024-05-30T04:36:02.145ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:26:02.142709894 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
34302024-05-30T04:36:02.145ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.238518ms iteration = 9
34312024-05-30T04:36:02.146ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
34322024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.240978ms iteration = 17
34332024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
34342024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.268188ms iteration = 10
34352024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34362024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34372024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34382024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34392024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34402024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
34412024-05-30T04:36:02.146ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.215078ms iteration = 10
34422024-05-30T04:36:02.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1384, tv_nsec: 256912213 } })
34432024-05-30T04:36:02.147ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34442024-05-30T04:36:02.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1384, tv_nsec: 257030364 } })
34452024-05-30T04:36:02.147ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34462024-05-30T04:36:02.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
34472024-05-30T04:36:02.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34482024-05-30T04:36:02.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34492024-05-30T04:36:02.148ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
34502024-05-30T04:36:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34512024-05-30T04:36:02.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34522024-05-30T04:36:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34532024-05-30T04:36:02.149ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
34542024-05-30T04:36:02.149ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
34552024-05-30T04:36:02.149ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
34562024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
34572024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34582024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
34592024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34602024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
34612024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34622024-05-30T04:36:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34632024-05-30T04:36:02.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34642024-05-30T04:36:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34652024-05-30T04:36:02.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34662024-05-30T04:36:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34672024-05-30T04:36:02.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1384, tv_nsec: 261309003 } })
34682024-05-30T04:36:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
34692024-05-30T04:36:02.152ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
34702024-05-30T04:36:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
34712024-05-30T04:36:02.152ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
34722024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34732024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34742024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34752024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
34762024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
34772024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
34782024-05-30T04:36:02.153ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
34792024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
34802024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34812024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
34822024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34832024-05-30T04:36:02.153ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34842024-05-30T04:36:02.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34852024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34862024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
34872024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34882024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34892024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34902024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34912024-05-30T04:36:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34922024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
34932024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
34942024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
34952024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
34962024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34972024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
34982024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34992024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35002024-05-30T04:36:02.156ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
35012024-05-30T04:36:02.156ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.85207ms iteration = 17
35022024-05-30T04:36:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35032024-05-30T04:36:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
35042024-05-30T04:36:02.157ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35052024-05-30T04:36:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35062024-05-30T04:36:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35072024-05-30T04:36:02.159ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35082024-05-30T04:36:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35092024-05-30T04:36:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35102024-05-30T04:36:02.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35112024-05-30T04:36:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35122024-05-30T04:36:02.160ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35132024-05-30T04:36:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35142024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
35152024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35162024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35172024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35182024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
35192024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35202024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
35212024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
35222024-05-30T04:36:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35232024-05-30T04:36:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35242024-05-30T04:36:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35252024-05-30T04:36:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35262024-05-30T04:36:02.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
35272024-05-30T04:36:02.163ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
35282024-05-30T04:36:02.164ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
35292024-05-30T04:36:02.164ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
35302024-05-30T04:36:02.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35312024-05-30T04:36:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
35322024-05-30T04:36:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35332024-05-30T04:36:02.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35342024-05-30T04:36:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
35352024-05-30T04:36:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
35362024-05-30T04:36:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
35372024-05-30T04:36:02.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35382024-05-30T04:36:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35392024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
35402024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.479275ms iteration = 49
35412024-05-30T04:36:02.169ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
35422024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 26.790668ms iteration = 18
35432024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
35442024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
35452024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
35462024-05-30T04:36:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.060801ms iteration = 10
35472024-05-30T04:36:02.170ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35482024-05-30T04:36:02.172ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
35492024-05-30T04:36:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 29.625241ms iteration = 10
35502024-05-30T04:36:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
35512024-05-30T04:36:02.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
35522024-05-30T04:36:02.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35532024-05-30T04:36:02.173ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35542024-05-30T04:36:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
35552024-05-30T04:36:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35562024-05-30T04:36:02.174ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
35572024-05-30T04:36:02.174ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
35582024-05-30T04:36:02.174ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
35592024-05-30T04:36:02.174ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 31.892177ms iteration = 10
35602024-05-30T04:36:02.176ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35612024-05-30T04:36:02.179ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35622024-05-30T04:36:02.179ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35632024-05-30T04:36:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35642024-05-30T04:36:02.187ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35652024-05-30T04:36:02.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35662024-05-30T04:36:02.195ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35672024-05-30T04:36:02.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35682024-05-30T04:36:02.197ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35692024-05-30T04:36:02.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
35702024-05-30T04:36:02.201ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35712024-05-30T04:36:02.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35722024-05-30T04:36:02.202ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 59.901979ms iteration = 17
35732024-05-30T04:36:02.203ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
35742024-05-30T04:36:02.203ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
35752024-05-30T04:36:02.203ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 53.794289ms iteration = 9
35762024-05-30T04:36:02.293ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
35772024-05-30T04:36:02.387ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
35782024-05-30T04:36:02.391ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
35792024-05-30T04:36:02.391ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35802024-05-30T04:36:02.391ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a012406c-2abc-433d-a333-fc1efd344aea", "content-length": "26", "date": "Thu, 30 May 2024 04:36:02 GMT"} })
35812024-05-30T04:36:02.391ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35822024-05-30T04:36:02.394ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
35832024-05-30T04:36:02.455ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
35842024-05-30T04:36:02.456ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b1d9b4fb-3743-4983-8e7c-caff832cb73c", "date": "Thu, 30 May 2024 04:36:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35852024-05-30T04:36:02.456ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35862024-05-30T04:36:02.457ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2748dc5a-4747-4682-a79b-1f2b34a4b041", "date": "Thu, 30 May 2024 04:36:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35872024-05-30T04:36:02.457ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35882024-05-30T04:36:02.459ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
35892024-05-30T04:36:02.459ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 316.274468ms iteration = 11
35902024-05-30T04:36:02.479ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
35912024-05-30T04:36:02.479ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35922024-05-30T04:36:02.479ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0c6ac42a-057d-4381-8131-77f31f2eeb31", "content-length": "26", "date": "Thu, 30 May 2024 04:36:02 GMT"} })
35932024-05-30T04:36:02.479ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35942024-05-30T04:36:02.481ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35952024-05-30T04:36:02.498ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
35962024-05-30T04:36:02.498ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35972024-05-30T04:36:02.499ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ba761c40-3e30-4e10-ae34-86341b1a53ad", "content-length": "26", "date": "Thu, 30 May 2024 04:36:02 GMT"} })
35982024-05-30T04:36:02.499ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35992024-05-30T04:36:02.501ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
36002024-05-30T04:36:02.551ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
36012024-05-30T04:36:02.551ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3df8467d-a9ef-4123-9087-adc342023c7d", "content-length": "691", "date": "Thu, 30 May 2024 04:36:02 GMT"} })
36022024-05-30T04:36:02.551ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
36032024-05-30T04:36:02.551ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
36042024-05-30T04:36:02.551ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
36052024-05-30T04:36:02.552ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 409.716044ms iteration = 17
36062024-05-30T04:36:02.611ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
36072024-05-30T04:36:02.612ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f8ffad4c-769c-45df-bf5e-53180cbbd638", "content-length": "2", "date": "Thu, 30 May 2024 04:36:01 GMT"} })
36082024-05-30T04:36:02.613ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36092024-05-30T04:36:02.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36102024-05-30T04:36:02.614ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 471.98455ms iteration = 17
36112024-05-30T04:36:02.671ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
36122024-05-30T04:36:02.712ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 570.072459ms iteration = 17
36132024-05-30T04:36:02.725ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36142024-05-30T04:36:02.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36152024-05-30T04:36:02.727ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
36162024-05-30T04:36:02.727ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36172024-05-30T04:36:02.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36182024-05-30T04:36:02.736ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
36192024-05-30T04:36:02.736ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "63439f52-3ce6-43a6-9775-abe875e6831c", "content-length": "121", "date": "Thu, 30 May 2024 04:36:01 GMT"} })
36202024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
36212024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
36222024-05-30T04:36:02.736ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
36232024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
36242024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
36252024-05-30T04:36:02.736ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
36262024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
36272024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
36282024-05-30T04:36:02.736ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
36292024-05-30T04:36:02.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36302024-05-30T04:36:02.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36312024-05-30T04:36:02.738ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
36322024-05-30T04:36:02.738ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "866f9887-b6a7-4d2e-a8b2-bfaad98e0f89", "content-length": "2", "date": "Thu, 30 May 2024 04:36:01 GMT"} })
36332024-05-30T04:36:02.738ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
36342024-05-30T04:36:02.739ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "324585ed-9b94-48b6-b4fe-ef0fcff41095", "content-length": "52", "date": "Thu, 30 May 2024 04:36:01 GMT"} })
36352024-05-30T04:36:02.739ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
36362024-05-30T04:36:02.739ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36372024-05-30T04:36:02.743ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
36382024-05-30T04:36:02.744ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3eac5c7d-c01b-4b85-af4e-7922061465fd", "date": "Thu, 30 May 2024 04:36:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36392024-05-30T04:36:02.744ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36402024-05-30T04:36:02.746ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ecd8d9a9-53dc-4c5d-b0a3-f1c15ac6108d", "date": "Thu, 30 May 2024 04:36:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36412024-05-30T04:36:02.746ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36422024-05-30T04:36:02.747ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "05d00419-555c-4020-9e30-80afe01d7db0", "date": "Thu, 30 May 2024 04:36:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36432024-05-30T04:36:02.747ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36442024-05-30T04:36:02.749ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
36452024-05-30T04:36:02.749ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 607.070954ms iteration = 11
36462024-05-30T04:36:02.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2f6e9e4d-eaa4-409b-b30c-da2078c31c3d", "date": "Thu, 30 May 2024 04:36:02 GMT"} })
36472024-05-30T04:36:02.784ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
36482024-05-30T04:36:02.784ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
36492024-05-30T04:36:02.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "76cdbeaa-47ff-4bcd-bcd9-731df3ef51b3", "content-length": "467", "date": "Thu, 30 May 2024 04:36:02 GMT"} })
36502024-05-30T04:36:02.784ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36512024-05-30T04:36:02.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36522024-05-30T04:36:02.788ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36532024-05-30T04:36:02.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36542024-05-30T04:36:02.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36552024-05-30T04:36:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36562024-05-30T04:36:02.792ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36572024-05-30T04:36:02.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36582024-05-30T04:36:02.798ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 655.556903ms iteration = 17
36592024-05-30T04:36:08.865ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1418 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 79cef291-e36f-4bd2-bf80-1c1782308c3c response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
36602024-05-30T04:36:12.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
36612024-05-30T04:36:12.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36622024-05-30T04:36:12.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36632024-05-30T04:36:12.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36642024-05-30T04:36:12.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36652024-05-30T04:36:12.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36662024-05-30T04:36:12.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36672024-05-30T04:36:12.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36682024-05-30T04:36:12.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36692024-05-30T04:36:12.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36702024-05-30T04:36:12.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
36712024-05-30T04:36:12.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36722024-05-30T04:36:12.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36732024-05-30T04:36:12.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
36742024-05-30T04:36:12.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.825573ms iteration = 50
36752024-05-30T04:36:18.864ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 802 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 2689044b-b066-4f2d-bc02-6a0c2819e72c response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
36762024-05-30T04:36:19.556ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:49365
36772024-05-30T04:36:19.556ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49365 req_id = 9fe73547-bb9f-42ce-8482-a35090e6a229 roles = RoleSet { roles: {} } uri = /metrics/producers
36782024-05-30T04:36:19.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49365 req_id = 9fe73547-bb9f-42ce-8482-a35090e6a229 resource = Database result = Ok(()) uri = /metrics/producers
36792024-05-30T04:36:19.588ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
36802024-05-30T04:36:19.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49365 req_id = 9fe73547-bb9f-42ce-8482-a35090e6a229 roles = RoleSet { roles: {} } uri = /metrics/producers
36812024-05-30T04:36:19.589ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49365 req_id = 9fe73547-bb9f-42ce-8482-a35090e6a229 resource = Database result = Ok(()) uri = /metrics/producers
36822024-05-30T04:36:19.594ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
36832024-05-30T04:36:19.594ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "20b81be0-0d22-4802-aaae-e6cd7853d189", "date": "Thu, 30 May 2024 04:36:19 GMT"} })
36842024-05-30T04:36:19.594ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 439e757e-803c-4f81-8f3d-8bd0f1f5d2cc
36852024-05-30T04:36:19.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 40419 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49365 req_id = 9fe73547-bb9f-42ce-8482-a35090e6a229 response_code = 201 uri = /metrics/producers
36862024-05-30T04:36:22.142ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
36872024-05-30T04:36:22.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36882024-05-30T04:36:22.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36892024-05-30T04:36:22.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36902024-05-30T04:36:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36912024-05-30T04:36:22.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36922024-05-30T04:36:22.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36932024-05-30T04:36:22.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36942024-05-30T04:36:22.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36952024-05-30T04:36:22.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36962024-05-30T04:36:22.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
36972024-05-30T04:36:22.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36982024-05-30T04:36:22.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36992024-05-30T04:36:22.160ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
37002024-05-30T04:36:22.160ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.176058ms iteration = 51
37012024-05-30T04:36:28.864ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 521 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 6dfd55b5-5f77-443e-94de-e5dc7e355ce9 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
37022024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
37032024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
37042024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37052024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
37062024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37072024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
37082024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
37092024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
37102024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
37112024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
37122024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37132024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37142024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
37152024-05-30T04:36:32.143ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
37162024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
37172024-05-30T04:36:32.143ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
37182024-05-30T04:36:32.143ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
37192024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
37202024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 12.9\u{b5}s iteration = 18
37212024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
37222024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37232024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37242024-05-30T04:36:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37252024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37262024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
37272024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37282024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37292024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37302024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
37312024-05-30T04:36:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37322024-05-30T04:36:32.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37332024-05-30T04:36:32.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37342024-05-30T04:36:32.146ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1414, tv_nsec: 255470646 } })
37352024-05-30T04:36:32.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37362024-05-30T04:36:32.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37372024-05-30T04:36:32.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
37382024-05-30T04:36:32.147ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
37392024-05-30T04:36:32.147ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.808243ms iteration = 18
37402024-05-30T04:36:32.147ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1414, tv_nsec: 256301786 } })
37412024-05-30T04:36:32.147ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37422024-05-30T04:36:32.148ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
37432024-05-30T04:36:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
37442024-05-30T04:36:32.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37452024-05-30T04:36:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37462024-05-30T04:36:32.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37472024-05-30T04:36:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37482024-05-30T04:36:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37492024-05-30T04:36:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37502024-05-30T04:36:32.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37512024-05-30T04:36:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
37522024-05-30T04:36:32.150ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37532024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37542024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37552024-05-30T04:36:32.151ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
37562024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
37572024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37582024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37592024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37602024-05-30T04:36:32.151ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1414, tv_nsec: 260498114 } })
37612024-05-30T04:36:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37622024-05-30T04:36:32.151ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
37632024-05-30T04:36:32.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37642024-05-30T04:36:32.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37652024-05-30T04:36:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37662024-05-30T04:36:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37672024-05-30T04:36:32.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37682024-05-30T04:36:32.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37692024-05-30T04:36:32.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37702024-05-30T04:36:32.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37712024-05-30T04:36:32.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37722024-05-30T04:36:32.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
37732024-05-30T04:36:32.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37742024-05-30T04:36:32.156ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
37752024-05-30T04:36:32.156ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 13.06657ms iteration = 18
37762024-05-30T04:36:32.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37772024-05-30T04:36:32.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37782024-05-30T04:36:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37792024-05-30T04:36:32.162ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
37802024-05-30T04:36:32.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.231211ms iteration = 52
37812024-05-30T04:36:32.162ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
37822024-05-30T04:36:32.162ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.175041ms iteration = 19
37832024-05-30T04:36:32.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37842024-05-30T04:36:32.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37852024-05-30T04:36:32.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37862024-05-30T04:36:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37872024-05-30T04:36:32.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37882024-05-30T04:36:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37892024-05-30T04:36:32.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37902024-05-30T04:36:32.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37912024-05-30T04:36:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37922024-05-30T04:36:32.167ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
37932024-05-30T04:36:32.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37942024-05-30T04:36:32.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37952024-05-30T04:36:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37962024-05-30T04:36:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
37972024-05-30T04:36:32.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37982024-05-30T04:36:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37992024-05-30T04:36:32.172ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.106625ms iteration = 18
38002024-05-30T04:36:32.247ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
38012024-05-30T04:36:32.247ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38022024-05-30T04:36:32.248ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5174b522-354c-4250-b7c9-9f7527548f3f", "content-length": "26", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38032024-05-30T04:36:32.248ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38042024-05-30T04:36:32.248ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
38052024-05-30T04:36:32.248ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38062024-05-30T04:36:32.249ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
38072024-05-30T04:36:32.250ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3185ba61-f277-4fea-b037-2f74e70e8b00", "content-length": "26", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38082024-05-30T04:36:32.250ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38092024-05-30T04:36:32.251ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
38102024-05-30T04:36:32.251ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38112024-05-30T04:36:32.251ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "84f84521-d3d5-49ef-b6ae-022d55101ea9", "content-length": "26", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38122024-05-30T04:36:32.251ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38132024-05-30T04:36:32.252ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
38142024-05-30T04:36:32.253ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
38152024-05-30T04:36:32.274ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
38162024-05-30T04:36:32.276ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3bebf9cf-3fb9-4e63-b294-189bf2533dc6", "content-length": "691", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38172024-05-30T04:36:32.276ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
38182024-05-30T04:36:32.276ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
38192024-05-30T04:36:32.276ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
38202024-05-30T04:36:32.277ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 133.905951ms iteration = 18
38212024-05-30T04:36:32.335ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
38222024-05-30T04:36:32.335ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "121c188b-b2f5-4187-a4a5-b84a2393d337", "content-length": "2", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38232024-05-30T04:36:32.337ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 193.653858ms iteration = 18
38242024-05-30T04:36:32.337ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
38252024-05-30T04:36:32.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
38262024-05-30T04:36:32.339ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 196.097286ms iteration = 18
38272024-05-30T04:36:32.395ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38282024-05-30T04:36:32.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38292024-05-30T04:36:32.397ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
38302024-05-30T04:36:32.397ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38312024-05-30T04:36:32.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38322024-05-30T04:36:32.404ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
38332024-05-30T04:36:32.404ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "871da3d8-5b48-43d0-bd06-e5cc27e1d10d", "content-length": "121", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38342024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
38352024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
38362024-05-30T04:36:32.404ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
38372024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
38382024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
38392024-05-30T04:36:32.404ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
38402024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
38412024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
38422024-05-30T04:36:32.404ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
38432024-05-30T04:36:32.404ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38442024-05-30T04:36:32.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38452024-05-30T04:36:32.406ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
38462024-05-30T04:36:32.406ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "34246d21-e6c3-4cf5-9a93-a9f0df91f36c", "content-length": "2", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38472024-05-30T04:36:32.406ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
38482024-05-30T04:36:32.407ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "12cc695e-3755-4c19-be51-e7d045b96029", "content-length": "52", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38492024-05-30T04:36:32.407ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
38502024-05-30T04:36:32.407ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
38512024-05-30T04:36:32.493ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a692a374-852c-449b-920c-d8cc72db524a", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38522024-05-30T04:36:32.493ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
38532024-05-30T04:36:32.493ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
38542024-05-30T04:36:32.497ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2836499a-22b6-4eb5-b9d0-abb0c17481f6", "content-length": "467", "date": "Thu, 30 May 2024 04:36:32 GMT"} })
38552024-05-30T04:36:32.497ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38562024-05-30T04:36:32.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38572024-05-30T04:36:32.500ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38582024-05-30T04:36:32.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38592024-05-30T04:36:32.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38602024-05-30T04:36:32.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38612024-05-30T04:36:32.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38622024-05-30T04:36:32.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38632024-05-30T04:36:32.510ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 367.250675ms iteration = 18
38642024-05-30T04:36:38.865ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 800 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 5cb91691-3dec-490e-a788-34127796511a response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
38652024-05-30T04:36:42.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
38662024-05-30T04:36:42.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38672024-05-30T04:36:42.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38682024-05-30T04:36:42.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38692024-05-30T04:36:42.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38702024-05-30T04:36:42.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38712024-05-30T04:36:42.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38722024-05-30T04:36:42.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38732024-05-30T04:36:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38742024-05-30T04:36:42.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38752024-05-30T04:36:42.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
38762024-05-30T04:36:42.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38772024-05-30T04:36:42.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38782024-05-30T04:36:42.159ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
38792024-05-30T04:36:42.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.109325ms iteration = 53
38802024-05-30T04:36:48.865ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 493 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = e3f83cc6-f91f-4eb7-ae7d-f03ecff52523 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
38812024-05-30T04:36:52.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
38822024-05-30T04:36:52.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38832024-05-30T04:36:52.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38842024-05-30T04:36:52.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38852024-05-30T04:36:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38862024-05-30T04:36:52.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38872024-05-30T04:36:52.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38882024-05-30T04:36:52.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38892024-05-30T04:36:52.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38902024-05-30T04:36:52.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38912024-05-30T04:36:52.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
38922024-05-30T04:36:52.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38932024-05-30T04:36:52.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38942024-05-30T04:36:52.160ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
38952024-05-30T04:36:52.160ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.991386ms iteration = 54
38962024-05-30T04:36:58.865ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 519 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 68496651-5571-4a9a-9783-edba55c7551c response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
38972024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
38982024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
38992024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39002024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
39012024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
39022024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39032024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
39042024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
39052024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
39062024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39072024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
39082024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
39092024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39102024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
39112024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
39122024-05-30T04:37:02.144ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:27:02.144119969 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
39132024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
39142024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
39152024-05-30T04:37:02.144ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
39162024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.52\u{b5}s iteration = 11
39172024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
39182024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
39192024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
39202024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
39212024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39222024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39232024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
39242024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39252024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
39262024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
39272024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39282024-05-30T04:37:02.144ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
39292024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
39302024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
39312024-05-30T04:37:02.144ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
39322024-05-30T04:37:02.144ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
39332024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.22\u{b5}s iteration = 19
39342024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39352024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
39362024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39372024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39382024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39392024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
39402024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39412024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39422024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39432024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39442024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39452024-05-30T04:37:02.144ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1444, tv_nsec: 253047609 } })
39462024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39472024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
39482024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
39492024-05-30T04:37:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39502024-05-30T04:37:02.145ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1444, tv_nsec: 254038531 } })
39512024-05-30T04:37:02.146ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
39522024-05-30T04:37:02.146ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.515449ms iteration = 11
39532024-05-30T04:37:02.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39542024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39552024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
39562024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.178216ms iteration = 11
39572024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39582024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39592024-05-30T04:37:02.147ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
39602024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.449ms iteration = 19
39612024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39622024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39632024-05-30T04:37:02.147ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:27:02.144119969 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
39642024-05-30T04:37:02.147ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.782434ms iteration = 10
39652024-05-30T04:37:02.148ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39662024-05-30T04:37:02.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39672024-05-30T04:37:02.148ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39682024-05-30T04:37:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39692024-05-30T04:37:02.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39702024-05-30T04:37:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39712024-05-30T04:37:02.150ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
39722024-05-30T04:37:02.150ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
39732024-05-30T04:37:02.150ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
39742024-05-30T04:37:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
39752024-05-30T04:37:02.151ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
39762024-05-30T04:37:02.151ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39772024-05-30T04:37:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39782024-05-30T04:37:02.151ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
39792024-05-30T04:37:02.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39802024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
39812024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39822024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39832024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39842024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39852024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39862024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39872024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39882024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39892024-05-30T04:37:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39902024-05-30T04:37:02.153ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1444, tv_nsec: 261423366 } })
39912024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39922024-05-30T04:37:02.153ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
39932024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39942024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39952024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39962024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39972024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
39982024-05-30T04:37:02.153ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
39992024-05-30T04:37:02.153ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 3.206937ms iteration = 10
40002024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40012024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40022024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40032024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40042024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40052024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40062024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40072024-05-30T04:37:02.154ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40082024-05-30T04:37:02.155ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
40092024-05-30T04:37:02.155ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
40102024-05-30T04:37:02.155ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
40112024-05-30T04:37:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40122024-05-30T04:37:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40132024-05-30T04:37:02.155ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1444, tv_nsec: 263996025 } })
40142024-05-30T04:37:02.155ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1444, tv_nsec: 264018215 } })
40152024-05-30T04:37:02.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40162024-05-30T04:37:02.155ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
40172024-05-30T04:37:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40182024-05-30T04:37:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40192024-05-30T04:37:02.156ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40202024-05-30T04:37:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40212024-05-30T04:37:02.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40222024-05-30T04:37:02.156ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
40232024-05-30T04:37:02.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40242024-05-30T04:37:02.158ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
40252024-05-30T04:37:02.158ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.481457ms iteration = 19
40262024-05-30T04:37:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40272024-05-30T04:37:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
40282024-05-30T04:37:02.159ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40292024-05-30T04:37:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40302024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
40312024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
40322024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
40332024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40342024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40352024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40362024-05-30T04:37:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40372024-05-30T04:37:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40382024-05-30T04:37:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40392024-05-30T04:37:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
40402024-05-30T04:37:02.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40412024-05-30T04:37:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40422024-05-30T04:37:02.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40432024-05-30T04:37:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40442024-05-30T04:37:02.162ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
40452024-05-30T04:37:02.162ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
40462024-05-30T04:37:02.162ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40472024-05-30T04:37:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40482024-05-30T04:37:02.163ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40492024-05-30T04:37:02.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40502024-05-30T04:37:02.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40512024-05-30T04:37:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40522024-05-30T04:37:02.165ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
40532024-05-30T04:37:02.165ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.612399ms iteration = 20
40542024-05-30T04:37:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40552024-05-30T04:37:02.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40562024-05-30T04:37:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40572024-05-30T04:37:02.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
40582024-05-30T04:37:02.166ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40592024-05-30T04:37:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40602024-05-30T04:37:02.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40612024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40622024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40632024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
40642024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
40652024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40662024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40672024-05-30T04:37:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40682024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
40692024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.352912ms iteration = 55
40702024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
40712024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40722024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
40732024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
40742024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
40752024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.715966ms iteration = 11
40762024-05-30T04:37:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40772024-05-30T04:37:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40782024-05-30T04:37:02.171ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40792024-05-30T04:37:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40802024-05-30T04:37:02.174ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40812024-05-30T04:37:02.174ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
40822024-05-30T04:37:02.175ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40832024-05-30T04:37:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40842024-05-30T04:37:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
40852024-05-30T04:37:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40862024-05-30T04:37:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40872024-05-30T04:37:02.177ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
40882024-05-30T04:37:02.177ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
40892024-05-30T04:37:02.177ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
40902024-05-30T04:37:02.177ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 33.456374ms iteration = 11
40912024-05-30T04:37:02.178ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40922024-05-30T04:37:02.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40932024-05-30T04:37:02.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40942024-05-30T04:37:02.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
40952024-05-30T04:37:02.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40962024-05-30T04:37:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40972024-05-30T04:37:02.195ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 51.094268ms iteration = 19
40982024-05-30T04:37:02.227ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
40992024-05-30T04:37:02.227ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 83.43302ms iteration = 11
41002024-05-30T04:37:02.334ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
41012024-05-30T04:37:02.340ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
41022024-05-30T04:37:02.340ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "11d7d0b1-6d79-4f92-b617-d3ef3cd7a1a2", "content-length": "691", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41032024-05-30T04:37:02.340ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
41042024-05-30T04:37:02.341ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
41052024-05-30T04:37:02.341ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
41062024-05-30T04:37:02.342ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 198.676466ms iteration = 19
41072024-05-30T04:37:02.348ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
41082024-05-30T04:37:02.348ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41092024-05-30T04:37:02.348ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d11c726e-5ab4-46b8-b2f6-1cd165b05661", "content-length": "26", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41102024-05-30T04:37:02.348ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41112024-05-30T04:37:02.350ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
41122024-05-30T04:37:02.354ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
41132024-05-30T04:37:02.354ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41142024-05-30T04:37:02.354ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a9ffb6cb-9d8f-4cfc-a0c9-823bc192bd79", "content-length": "26", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41152024-05-30T04:37:02.354ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41162024-05-30T04:37:02.357ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41172024-05-30T04:37:02.392ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
41182024-05-30T04:37:02.457ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
41192024-05-30T04:37:02.457ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41202024-05-30T04:37:02.458ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3081576e-73b5-4540-9e7c-35a78e721d51", "content-length": "26", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41212024-05-30T04:37:02.458ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41222024-05-30T04:37:02.459ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41232024-05-30T04:37:02.466ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
41242024-05-30T04:37:02.467ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6a778013-71a2-45b7-879e-3cbec4f7ce76", "date": "Thu, 30 May 2024 04:37:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41252024-05-30T04:37:02.467ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41262024-05-30T04:37:02.468ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ec2a2bcf-1eb8-4de8-ad3d-2c40e0327591", "date": "Thu, 30 May 2024 04:37:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41272024-05-30T04:37:02.468ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41282024-05-30T04:37:02.470ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
41292024-05-30T04:37:02.470ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 326.691919ms iteration = 12
41302024-05-30T04:37:02.548ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
41312024-05-30T04:37:02.549ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e8aa687a-8610-4010-9643-747a70966a6b", "content-length": "2", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41322024-05-30T04:37:02.550ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
41332024-05-30T04:37:02.551ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
41342024-05-30T04:37:02.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
41352024-05-30T04:37:02.553ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 409.115538ms iteration = 19
41362024-05-30T04:37:02.597ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 453.022043ms iteration = 19
41372024-05-30T04:37:02.614ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
41382024-05-30T04:37:02.615ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c65a44f7-eb91-44a5-9743-ed53eac6bf85", "date": "Thu, 30 May 2024 04:37:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41392024-05-30T04:37:02.615ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41402024-05-30T04:37:02.617ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a18de2f8-e9ca-467f-bdc3-6f1059f737ce", "date": "Thu, 30 May 2024 04:37:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41412024-05-30T04:37:02.617ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41422024-05-30T04:37:02.618ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "15cc1735-34b3-4de1-b86d-5e786ff6aeab", "date": "Thu, 30 May 2024 04:37:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41432024-05-30T04:37:02.618ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41442024-05-30T04:37:02.620ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
41452024-05-30T04:37:02.620ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 476.783505ms iteration = 12
41462024-05-30T04:37:02.626ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41472024-05-30T04:37:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41482024-05-30T04:37:02.628ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
41492024-05-30T04:37:02.628ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41502024-05-30T04:37:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41512024-05-30T04:37:02.635ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
41522024-05-30T04:37:02.636ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "20702ca0-3a97-4c55-95ea-2e87c0290387", "content-length": "121", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41532024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
41542024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
41552024-05-30T04:37:02.636ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
41562024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
41572024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
41582024-05-30T04:37:02.636ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
41592024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
41602024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
41612024-05-30T04:37:02.636ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
41622024-05-30T04:37:02.636ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41632024-05-30T04:37:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41642024-05-30T04:37:02.638ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
41652024-05-30T04:37:02.639ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "28341ca2-9980-40f3-8ea6-c02b5d75bfbd", "content-length": "2", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41662024-05-30T04:37:02.639ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
41672024-05-30T04:37:02.639ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cdf4fc92-fbdb-4573-80a4-acbd259b4fd9", "content-length": "52", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41682024-05-30T04:37:02.639ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
41692024-05-30T04:37:02.639ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41702024-05-30T04:37:02.685ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9c4e6d65-951d-4246-bcf4-cb0e687d5774", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41712024-05-30T04:37:02.685ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
41722024-05-30T04:37:02.685ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
41732024-05-30T04:37:02.685ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4dcec405-6c2d-4073-8736-704ef1b23762", "content-length": "467", "date": "Thu, 30 May 2024 04:37:02 GMT"} })
41742024-05-30T04:37:02.685ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41752024-05-30T04:37:02.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41762024-05-30T04:37:02.688ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41772024-05-30T04:37:02.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41782024-05-30T04:37:02.690ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41792024-05-30T04:37:02.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41802024-05-30T04:37:02.692ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41812024-05-30T04:37:02.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41822024-05-30T04:37:02.699ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 555.35498ms iteration = 19
41832024-05-30T04:37:08.866ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 811 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = aa232b08-359e-47f7-ba3b-8029cdaaa1ff response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
41842024-05-30T04:37:09.101ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::2]:33511
41852024-05-30T04:37:09.101ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33511 req_id = 8d18bd6f-ba16-4392-887b-f15cf511440e roles = RoleSet { roles: {} } uri = /metrics/producers
41862024-05-30T04:37:09.102ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33511 req_id = 8d18bd6f-ba16-4392-887b-f15cf511440e resource = Database result = Ok(()) uri = /metrics/producers
41872024-05-30T04:37:09.134ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
41882024-05-30T04:37:09.134ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33511 req_id = 8d18bd6f-ba16-4392-887b-f15cf511440e roles = RoleSet { roles: {} } uri = /metrics/producers
41892024-05-30T04:37:09.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33511 req_id = 8d18bd6f-ba16-4392-887b-f15cf511440e resource = Database result = Ok(()) uri = /metrics/producers
41902024-05-30T04:37:09.139ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
41912024-05-30T04:37:09.140ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "78f5478b-13cf-4f1a-bbe9-76d34f762cdd", "date": "Thu, 30 May 2024 04:37:09 GMT"} })
41922024-05-30T04:37:09.140ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = 0cf65393-f6a2-480b-8f0b-78c9543ba992
41932024-05-30T04:37:09.142ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 40602 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:33511 req_id = 8d18bd6f-ba16-4392-887b-f15cf511440e response_code = 201 uri = /metrics/producers
41942024-05-30T04:37:12.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
41952024-05-30T04:37:12.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41962024-05-30T04:37:12.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
41972024-05-30T04:37:12.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41982024-05-30T04:37:12.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41992024-05-30T04:37:12.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42002024-05-30T04:37:12.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42012024-05-30T04:37:12.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42022024-05-30T04:37:12.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42032024-05-30T04:37:12.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42042024-05-30T04:37:12.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
42052024-05-30T04:37:12.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42062024-05-30T04:37:12.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42072024-05-30T04:37:12.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
42082024-05-30T04:37:12.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.945966ms iteration = 56
42092024-05-30T04:37:18.866ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1128 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = d302fe3b-6552-4788-b45e-bc613dbff7ae response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
42102024-05-30T04:37:22.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
42112024-05-30T04:37:22.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42122024-05-30T04:37:22.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42132024-05-30T04:37:22.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42142024-05-30T04:37:22.148ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42152024-05-30T04:37:22.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42162024-05-30T04:37:22.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42172024-05-30T04:37:22.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42182024-05-30T04:37:22.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42192024-05-30T04:37:22.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42202024-05-30T04:37:22.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
42212024-05-30T04:37:22.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42222024-05-30T04:37:22.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42232024-05-30T04:37:22.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
42242024-05-30T04:37:22.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.664123ms iteration = 57
42252024-05-30T04:37:28.866ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 811 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = 4a802078-ff17-4b39-96f4-9a51c546e983 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
42262024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
42272024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
42282024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42292024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42302024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
42312024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
42322024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
42332024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42342024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
42352024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
42362024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
42372024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42382024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
42392024-05-30T04:37:32.144ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
42402024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
42412024-05-30T04:37:32.144ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
42422024-05-30T04:37:32.144ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
42432024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
42442024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.31\u{b5}s iteration = 20
42452024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
42462024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
42472024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42482024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42492024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42502024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
42512024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42522024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
42532024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
42542024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
42552024-05-30T04:37:32.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42562024-05-30T04:37:32.146ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42572024-05-30T04:37:32.146ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42582024-05-30T04:37:32.146ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1474, tv_nsec: 254804320 } })
42592024-05-30T04:37:32.147ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
42602024-05-30T04:37:32.147ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
42612024-05-30T04:37:32.147ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.245187ms iteration = 20
42622024-05-30T04:37:32.147ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42632024-05-30T04:37:32.148ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1474, tv_nsec: 256011874 } })
42642024-05-30T04:37:32.148ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42652024-05-30T04:37:32.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42662024-05-30T04:37:32.148ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
42672024-05-30T04:37:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42682024-05-30T04:37:32.150ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42692024-05-30T04:37:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
42702024-05-30T04:37:32.150ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
42712024-05-30T04:37:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42722024-05-30T04:37:32.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
42732024-05-30T04:37:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
42742024-05-30T04:37:32.151ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42752024-05-30T04:37:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42762024-05-30T04:37:32.151ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
42772024-05-30T04:37:32.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42782024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
42792024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42802024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42812024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42822024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42832024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42842024-05-30T04:37:32.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42852024-05-30T04:37:32.153ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
42862024-05-30T04:37:32.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
42872024-05-30T04:37:32.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42882024-05-30T04:37:32.153ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1474, tv_nsec: 261825680 } })
42892024-05-30T04:37:32.154ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
42902024-05-30T04:37:32.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42912024-05-30T04:37:32.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42922024-05-30T04:37:32.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
42932024-05-30T04:37:32.155ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
42942024-05-30T04:37:32.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
42952024-05-30T04:37:32.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42962024-05-30T04:37:32.156ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
42972024-05-30T04:37:32.156ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.071399ms iteration = 20
42982024-05-30T04:37:32.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
42992024-05-30T04:37:32.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43002024-05-30T04:37:32.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43012024-05-30T04:37:32.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43022024-05-30T04:37:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43032024-05-30T04:37:32.163ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:174
43042024-05-30T04:37:32.163ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.226781ms iteration = 21
43052024-05-30T04:37:32.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43062024-05-30T04:37:32.164ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
43072024-05-30T04:37:32.164ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.104161ms iteration = 58
43082024-05-30T04:37:32.165ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43092024-05-30T04:37:32.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43102024-05-30T04:37:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43112024-05-30T04:37:32.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43122024-05-30T04:37:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43132024-05-30T04:37:32.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43142024-05-30T04:37:32.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43152024-05-30T04:37:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43162024-05-30T04:37:32.168ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
43172024-05-30T04:37:32.168ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43182024-05-30T04:37:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43192024-05-30T04:37:32.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43202024-05-30T04:37:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
43212024-05-30T04:37:32.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43222024-05-30T04:37:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43232024-05-30T04:37:32.175ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.464302ms iteration = 20
43242024-05-30T04:37:32.245ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
43252024-05-30T04:37:32.245ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43262024-05-30T04:37:32.245ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d79e637b-bd29-4a04-a442-1b17724fe433", "content-length": "26", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43272024-05-30T04:37:32.245ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43282024-05-30T04:37:32.247ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43292024-05-30T04:37:32.254ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
43302024-05-30T04:37:32.254ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43312024-05-30T04:37:32.254ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0b349d77-deb2-47c9-86af-25a5057c913d", "content-length": "26", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43322024-05-30T04:37:32.254ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43332024-05-30T04:37:32.254ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
43342024-05-30T04:37:32.254ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
43352024-05-30T04:37:32.255ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7a46ae66-9d1c-4df7-bee7-80d29cce7861", "content-length": "26", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43362024-05-30T04:37:32.255ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
43372024-05-30T04:37:32.256ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
43382024-05-30T04:37:32.257ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
43392024-05-30T04:37:32.295ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
43402024-05-30T04:37:32.296ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b45a0a5a-e350-4057-b6c6-c658849202ea", "content-length": "691", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43412024-05-30T04:37:32.296ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
43422024-05-30T04:37:32.296ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
43432024-05-30T04:37:32.296ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
43442024-05-30T04:37:32.298ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 154.037202ms iteration = 20
43452024-05-30T04:37:32.331ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
43462024-05-30T04:37:32.332ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "87e64473-3630-4cb2-bfda-30765a1de3eb", "content-length": "2", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43472024-05-30T04:37:32.333ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
43482024-05-30T04:37:32.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
43492024-05-30T04:37:32.335ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 191.058908ms iteration = 20
43502024-05-30T04:37:32.347ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 202.860124ms iteration = 20
43512024-05-30T04:37:32.403ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43522024-05-30T04:37:32.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43532024-05-30T04:37:32.405ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
43542024-05-30T04:37:32.405ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43552024-05-30T04:37:32.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43562024-05-30T04:37:32.414ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
43572024-05-30T04:37:32.415ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d50eb674-7562-40c1-894e-5be9f53b5cf5", "content-length": "121", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43582024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
43592024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
43602024-05-30T04:37:32.415ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
43612024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
43622024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
43632024-05-30T04:37:32.415ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
43642024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
43652024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
43662024-05-30T04:37:32.415ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
43672024-05-30T04:37:32.415ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43682024-05-30T04:37:32.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43692024-05-30T04:37:32.417ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
43702024-05-30T04:37:32.417ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0c2af683-79e0-4d10-9aee-40baf60e64a3", "content-length": "2", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43712024-05-30T04:37:32.417ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
43722024-05-30T04:37:32.417ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b0665b71-1410-4d8b-b97e-a941e52c15b4", "content-length": "52", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43732024-05-30T04:37:32.417ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
43742024-05-30T04:37:32.417ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
43752024-05-30T04:37:32.462ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4fb50253-5a2b-43c5-ba2a-4442df7800ec", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43762024-05-30T04:37:32.462ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
43772024-05-30T04:37:32.463ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
43782024-05-30T04:37:32.466ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "748cb58f-df39-46a1-bc85-0cd3a7eebf32", "content-length": "467", "date": "Thu, 30 May 2024 04:37:32 GMT"} })
43792024-05-30T04:37:32.466ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43802024-05-30T04:37:32.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43812024-05-30T04:37:32.469ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43822024-05-30T04:37:32.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43832024-05-30T04:37:32.472ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43842024-05-30T04:37:32.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43852024-05-30T04:37:32.473ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43862024-05-30T04:37:32.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43872024-05-30T04:37:32.480ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 335.761684ms iteration = 20
43882024-05-30T04:37:38.865ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 450 local_addr = [fd00:1122:3344:101::c]:56745 method = GET remote_addr = [fd00:1122:3344:101::d]:49522 req_id = f5518e43-9f92-489b-80fd-5a9b244ec477 response_code = 200 uri = /31c86263-9349-4603-aaa0-6dc44827df57
43892024-05-30T04:37:42.144ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
43902024-05-30T04:37:42.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43912024-05-30T04:37:42.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43922024-05-30T04:37:42.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43932024-05-30T04:37:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43942024-05-30T04:37:42.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43952024-05-30T04:37:42.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43962024-05-30T04:37:42.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43972024-05-30T04:37:42.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43982024-05-30T04:37:42.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43992024-05-30T04:37:42.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
44002024-05-30T04:37:42.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44012024-05-30T04:37:42.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44022024-05-30T04:37:42.161ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
44032024-05-30T04:37:42.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.382669ms iteration = 59