Max level shown:
12024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
22024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
32024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
42024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
52024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
62024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
72024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
82024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
92024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
112024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
122024-05-30T04:30:02.149ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 264995394 } })
132024-05-30T04:30:02.149ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
142024-05-30T04:30:02.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
152024-05-30T04:30:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
162024-05-30T04:30:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
172024-05-30T04:30:02.151ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 266577492 } })
182024-05-30T04:30:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192024-05-30T04:30:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202024-05-30T04:30:02.152ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
212024-05-30T04:30:02.152ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.634432ms iteration = 3
222024-05-30T04:30:02.156ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
232024-05-30T04:30:02.156ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.403365ms iteration = 3
242024-05-30T04:30:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
252024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
262024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
272024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
282024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
292024-05-30T04:30:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
302024-05-30T04:30:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
312024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
322024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
332024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
342024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
352024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
362024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 5 reason = Timeout
372024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
382024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
392024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
402024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
412024-05-30T04:30:02.160ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
422024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.751\u{b5}s iteration = 4
432024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
442024-05-30T04:30:02.160ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:20:02.160514724 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
452024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
462024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
472024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
482024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
492024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
502024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
512024-05-30T04:30:02.160ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
522024-05-30T04:30:02.160ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
532024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
542024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
552024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
562024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
572024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
582024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
592024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
602024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
612024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
622024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
632024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
642024-05-30T04:30:02.161ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
652024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
662024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
672024-05-30T04:30:02.161ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
682024-05-30T04:30:02.161ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
692024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.53\u{b5}s iteration = 5
702024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
712024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
722024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
732024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
742024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
752024-05-30T04:30:02.161ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
762024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
772024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
782024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
792024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
802024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
812024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
822024-05-30T04:30:02.162ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
832024-05-30T04:30:02.162ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.298696ms iteration = 5
842024-05-30T04:30:02.164ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:20:02.160514724 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
852024-05-30T04:30:02.164ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.592342ms iteration = 3
862024-05-30T04:30:02.165ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 280839426 } })
872024-05-30T04:30:02.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
882024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
892024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
902024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
912024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
922024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
932024-05-30T04:30:02.167ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 282898560 } })
942024-05-30T04:30:02.167ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
952024-05-30T04:30:02.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
962024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
972024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
982024-05-30T04:30:02.170ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
992024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1002024-05-30T04:30:02.170ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1012024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
1022024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1032024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1042024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1052024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1062024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1072024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1082024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1092024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1102024-05-30T04:30:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1112024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1122024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
1132024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1142024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1152024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1162024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1172024-05-30T04:30:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1182024-05-30T04:30:02.174ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1192024-05-30T04:30:02.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1202024-05-30T04:30:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1212024-05-30T04:30:02.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1222024-05-30T04:30:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1232024-05-30T04:30:02.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1242024-05-30T04:30:02.176ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1252024-05-30T04:30:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1262024-05-30T04:30:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1272024-05-30T04:30:02.179ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1282024-05-30T04:30:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1292024-05-30T04:30:02.182ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
1302024-05-30T04:30:02.182ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
1312024-05-30T04:30:02.182ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.013002ms iteration = 5
1322024-05-30T04:30:02.183ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1332024-05-30T04:30:02.183ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1342024-05-30T04:30:02.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1352024-05-30T04:30:02.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1362024-05-30T04:30:02.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1372024-05-30T04:30:02.186ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1024, tv_nsec: 301255701 } })
1382024-05-30T04:30:02.186ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
1392024-05-30T04:30:02.187ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
1402024-05-30T04:30:02.187ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.018749ms iteration = 5
1412024-05-30T04:30:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1422024-05-30T04:30:02.189ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1432024-05-30T04:30:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1442024-05-30T04:30:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
1452024-05-30T04:30:02.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1462024-05-30T04:30:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1472024-05-30T04:30:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1482024-05-30T04:30:02.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1492024-05-30T04:30:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1502024-05-30T04:30:02.192ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1512024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1522024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
1532024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1542024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
1552024-05-30T04:30:02.193ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 44.142698ms iteration = 3
1562024-05-30T04:30:02.194ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1572024-05-30T04:30:02.197ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1582024-05-30T04:30:02.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1592024-05-30T04:30:02.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
1602024-05-30T04:30:02.201ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
1612024-05-30T04:30:02.201ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 40.894991ms iteration = 13
1622024-05-30T04:30:02.201ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1632024-05-30T04:30:02.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1642024-05-30T04:30:02.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1652024-05-30T04:30:02.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1662024-05-30T04:30:02.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1672024-05-30T04:30:02.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1682024-05-30T04:30:02.212ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1692024-05-30T04:30:02.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1702024-05-30T04:30:02.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1712024-05-30T04:30:02.215ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1722024-05-30T04:30:02.215ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1732024-05-30T04:30:02.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1742024-05-30T04:30:02.219ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1752024-05-30T04:30:02.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1762024-05-30T04:30:02.221ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1772024-05-30T04:30:02.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
1782024-05-30T04:30:02.225ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1792024-05-30T04:30:02.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1802024-05-30T04:30:02.227ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
1812024-05-30T04:30:02.227ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
1822024-05-30T04:30:02.227ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 65.597504ms iteration = 3
1832024-05-30T04:30:02.227ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 65.849318ms iteration = 5
1842024-05-30T04:30:02.237ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
1852024-05-30T04:30:02.237ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
1862024-05-30T04:30:02.237ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1872024-05-30T04:30:02.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1882024-05-30T04:30:02.238ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1892024-05-30T04:30:02.241ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1902024-05-30T04:30:02.241ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1912024-05-30T04:30:02.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1922024-05-30T04:30:02.243ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
1932024-05-30T04:30:02.243ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
1942024-05-30T04:30:02.243ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
1952024-05-30T04:30:02.243ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 94.628278ms iteration = 3
1962024-05-30T04:30:02.252ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
1972024-05-30T04:30:02.255ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 94.8169ms iteration = 3
1982024-05-30T04:30:02.292ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
1992024-05-30T04:30:02.339ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
2002024-05-30T04:30:02.339ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2012024-05-30T04:30:02.339ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "54f26ab2-0e0f-4c17-a212-3a2c78c48db3", "content-length": "26", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2022024-05-30T04:30:02.339ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2032024-05-30T04:30:02.341ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2042024-05-30T04:30:02.406ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
2052024-05-30T04:30:02.488ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
2062024-05-30T04:30:02.520ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
2072024-05-30T04:30:02.520ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2082024-05-30T04:30:02.520ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fef4db61-7624-45d5-b798-9b040523f710", "content-length": "26", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2092024-05-30T04:30:02.520ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2102024-05-30T04:30:02.523ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
2112024-05-30T04:30:02.527ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
2122024-05-30T04:30:02.527ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2132024-05-30T04:30:02.527ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cad210d0-5b36-4ed0-8434-f93d5c2ecaad", "content-length": "26", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2142024-05-30T04:30:02.527ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2152024-05-30T04:30:02.529ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
2162024-05-30T04:30:02.570ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2172024-05-30T04:30:02.570ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dddfcb3b-0dc5-4f8b-ab40-1f10a4acbfed", "content-length": "691", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2182024-05-30T04:30:02.570ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
2192024-05-30T04:30:02.570ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
2202024-05-30T04:30:02.570ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
2212024-05-30T04:30:02.572ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 410.769586ms iteration = 5
2222024-05-30T04:30:02.665ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
2232024-05-30T04:30:02.667ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "31bb2085-e502-4e57-97d4-f3902894c686", "content-length": "1", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2242024-05-30T04:30:02.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
2252024-05-30T04:30:02.667ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a5b26ea4-c8d7-4178-b652-7a036cb8f90f", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2262024-05-30T04:30:02.667ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2272024-05-30T04:30:02.669ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2282024-05-30T04:30:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2292024-05-30T04:30:02.670ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e4055323-4600-4d6b-884c-0bdd943d3428", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2302024-05-30T04:30:02.670ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2312024-05-30T04:30:02.672ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 511.740448ms iteration = 5
2322024-05-30T04:30:02.672ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
2332024-05-30T04:30:02.672ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 512.166733ms iteration = 5
2342024-05-30T04:30:02.673ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
2352024-05-30T04:30:02.675ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4622605d-7558-4b65-90c0-8d145cb19a1d", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2362024-05-30T04:30:02.675ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2372024-05-30T04:30:02.677ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b992fc36-96a6-4519-b412-83db325f6c11", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2382024-05-30T04:30:02.677ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2392024-05-30T04:30:02.679ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "446c1b8c-557f-4090-a9ab-a3b1f5da8510", "date": "Thu, 30 May 2024 04:30:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2402024-05-30T04:30:02.679ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2412024-05-30T04:30:02.681ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
2422024-05-30T04:30:02.681ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 532.234104ms iteration = 5
2432024-05-30T04:30:02.686ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 526.143674ms iteration = 5
2442024-05-30T04:30:02.703ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 191 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 81018097-67e7-4e8d-a842-d4823f48fb56 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
2452024-05-30T04:30:02.771ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2462024-05-30T04:30:02.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2472024-05-30T04:30:02.774ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2482024-05-30T04:30:02.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2492024-05-30T04:30:02.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2502024-05-30T04:30:02.784ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2512024-05-30T04:30:02.785ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "71f0969f-758f-46dd-bc68-2065b7b1e540", "content-length": "121", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2522024-05-30T04:30:02.785ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
2532024-05-30T04:30:02.785ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
2542024-05-30T04:30:02.786ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2552024-05-30T04:30:02.786ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
2562024-05-30T04:30:02.786ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
2572024-05-30T04:30:02.786ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2582024-05-30T04:30:02.786ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
2592024-05-30T04:30:02.786ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
2602024-05-30T04:30:02.786ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
2612024-05-30T04:30:02.786ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2622024-05-30T04:30:02.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2632024-05-30T04:30:02.789ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2642024-05-30T04:30:02.789ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "58984045-4d3d-4f1b-9fe3-ab2b0d4445aa", "content-length": "2", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2652024-05-30T04:30:02.789ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2662024-05-30T04:30:02.790ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fbeadd9f-0e71-4fc6-8dc7-f149b21fbfb7", "content-length": "52", "date": "Thu, 30 May 2024 04:30:01 GMT"} })
2672024-05-30T04:30:02.790ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
2682024-05-30T04:30:02.790ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2692024-05-30T04:30:02.909ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e29d517c-90a1-46d4-910e-b24daa5362d2", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2702024-05-30T04:30:02.909ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
2712024-05-30T04:30:02.909ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2722024-05-30T04:30:02.914ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1669193c-d924-4d8f-9e0e-3926cf9ce812", "content-length": "467", "date": "Thu, 30 May 2024 04:30:02 GMT"} })
2732024-05-30T04:30:02.914ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2742024-05-30T04:30:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2752024-05-30T04:30:02.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2762024-05-30T04:30:02.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2772024-05-30T04:30:02.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2782024-05-30T04:30:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2792024-05-30T04:30:02.922ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2802024-05-30T04:30:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2812024-05-30T04:30:02.929ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 767.900806ms iteration = 5
2822024-05-30T04:30:12.160ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
2832024-05-30T04:30:12.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2842024-05-30T04:30:12.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2852024-05-30T04:30:12.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2862024-05-30T04:30:12.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2872024-05-30T04:30:12.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2882024-05-30T04:30:12.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2892024-05-30T04:30:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2902024-05-30T04:30:12.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2912024-05-30T04:30:12.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2922024-05-30T04:30:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
2932024-05-30T04:30:12.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2942024-05-30T04:30:12.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2952024-05-30T04:30:12.175ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
2962024-05-30T04:30:12.175ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.918371ms iteration = 14
2972024-05-30T04:30:12.703ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 149 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 7f67b5e0-901f-4b63-8cb5-12a034e2c945 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
2982024-05-30T04:30:22.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
2992024-05-30T04:30:22.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3002024-05-30T04:30:22.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3012024-05-30T04:30:22.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3022024-05-30T04:30:22.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3032024-05-30T04:30:22.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3042024-05-30T04:30:22.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3052024-05-30T04:30:22.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3062024-05-30T04:30:22.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3072024-05-30T04:30:22.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3082024-05-30T04:30:22.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
3092024-05-30T04:30:22.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3102024-05-30T04:30:22.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3112024-05-30T04:30:22.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
3122024-05-30T04:30:22.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.867284ms iteration = 15
3132024-05-30T04:30:22.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 173 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 21e5f1a4-d863-47f7-b8c0-7c9643b9139c response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
3142024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
3152024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
3162024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3172024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3182024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
3192024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
3202024-05-30T04:30:32.160ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
3212024-05-30T04:30:32.160ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
3222024-05-30T04:30:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3232024-05-30T04:30:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3242024-05-30T04:30:32.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3252024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
3262024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
3272024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3282024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3292024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
3302024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
3312024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3322024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
3332024-05-30T04:30:32.162ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
3342024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3352024-05-30T04:30:32.162ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
3362024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.96\u{b5}s iteration = 6
3372024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
3382024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3392024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3402024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3412024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3422024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3432024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3442024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3452024-05-30T04:30:32.162ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3462024-05-30T04:30:32.162ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.023624ms iteration = 6
3472024-05-30T04:30:32.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3482024-05-30T04:30:32.164ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1054, tv_nsec: 279109096 } })
3492024-05-30T04:30:32.164ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3502024-05-30T04:30:32.164ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1054, tv_nsec: 279252068 } })
3512024-05-30T04:30:32.164ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
3522024-05-30T04:30:32.164ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3532024-05-30T04:30:32.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3542024-05-30T04:30:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3552024-05-30T04:30:32.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3562024-05-30T04:30:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3572024-05-30T04:30:32.166ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
3582024-05-30T04:30:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3592024-05-30T04:30:32.167ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3602024-05-30T04:30:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3612024-05-30T04:30:32.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3622024-05-30T04:30:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3632024-05-30T04:30:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3642024-05-30T04:30:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3652024-05-30T04:30:32.168ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3662024-05-30T04:30:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3672024-05-30T04:30:32.168ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
3682024-05-30T04:30:32.168ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
3692024-05-30T04:30:32.168ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 6.636157ms iteration = 6
3702024-05-30T04:30:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3712024-05-30T04:30:32.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3722024-05-30T04:30:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3732024-05-30T04:30:32.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3742024-05-30T04:30:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3752024-05-30T04:30:32.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3762024-05-30T04:30:32.170ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1054, tv_nsec: 285725182 } })
3772024-05-30T04:30:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3782024-05-30T04:30:32.171ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
3792024-05-30T04:30:32.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3802024-05-30T04:30:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
3812024-05-30T04:30:32.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3822024-05-30T04:30:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3832024-05-30T04:30:32.180ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
3842024-05-30T04:30:32.180ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.635486ms iteration = 16
3852024-05-30T04:30:32.189ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3862024-05-30T04:30:32.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3872024-05-30T04:30:32.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3882024-05-30T04:30:32.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3892024-05-30T04:30:32.192ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3902024-05-30T04:30:32.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3912024-05-30T04:30:32.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3922024-05-30T04:30:32.194ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3932024-05-30T04:30:32.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3942024-05-30T04:30:32.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3952024-05-30T04:30:32.197ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3962024-05-30T04:30:32.197ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3972024-05-30T04:30:32.199ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3982024-05-30T04:30:32.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3992024-05-30T04:30:32.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4002024-05-30T04:30:32.200ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
4012024-05-30T04:30:32.200ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4022024-05-30T04:30:32.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4032024-05-30T04:30:32.202ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 40.076001ms iteration = 6
4042024-05-30T04:30:32.232ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
4052024-05-30T04:30:32.232ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 70.718544ms iteration = 6
4062024-05-30T04:30:32.277ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
4072024-05-30T04:30:32.277ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4082024-05-30T04:30:32.278ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "650ca19c-011b-41f5-8bf3-19fa933b48c5", "content-length": "26", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4092024-05-30T04:30:32.278ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4102024-05-30T04:30:32.279ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
4112024-05-30T04:30:32.423ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
4122024-05-30T04:30:32.423ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4132024-05-30T04:30:32.423ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
4142024-05-30T04:30:32.423ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4152024-05-30T04:30:32.424ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b1b1a205-d170-4fa3-a125-9ba7f3bf400d", "content-length": "26", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4162024-05-30T04:30:32.424ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4172024-05-30T04:30:32.424ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bccb2cf9-acc6-425d-8d5a-a07f10cb142b", "content-length": "26", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4182024-05-30T04:30:32.424ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4192024-05-30T04:30:32.427ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4202024-05-30T04:30:32.427ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4212024-05-30T04:30:32.491ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
4222024-05-30T04:30:32.529ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "814eb790-9238-4788-bf0f-5671d513a1cf", "content-length": "691", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4232024-05-30T04:30:32.529ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
4242024-05-30T04:30:32.529ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
4252024-05-30T04:30:32.529ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
4262024-05-30T04:30:32.531ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 369.538312ms iteration = 6
4272024-05-30T04:30:32.557ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
4282024-05-30T04:30:32.558ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0a960089-550d-4013-8ddc-875bcdf10642", "content-length": "1", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4292024-05-30T04:30:32.559ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4302024-05-30T04:30:32.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4312024-05-30T04:30:32.561ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 400.6834ms iteration = 6
4322024-05-30T04:30:32.614ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 453.793232ms iteration = 6
4332024-05-30T04:30:32.637ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4342024-05-30T04:30:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4352024-05-30T04:30:32.645ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4362024-05-30T04:30:32.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4372024-05-30T04:30:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4382024-05-30T04:30:32.653ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
4392024-05-30T04:30:32.653ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5b8ef201-581d-41f6-aff7-aee60522e2e2", "content-length": "121", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4402024-05-30T04:30:32.653ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
4412024-05-30T04:30:32.653ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
4422024-05-30T04:30:32.654ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4432024-05-30T04:30:32.654ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
4442024-05-30T04:30:32.654ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
4452024-05-30T04:30:32.654ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4462024-05-30T04:30:32.654ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
4472024-05-30T04:30:32.654ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
4482024-05-30T04:30:32.654ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
4492024-05-30T04:30:32.654ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4502024-05-30T04:30:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4512024-05-30T04:30:32.656ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
4522024-05-30T04:30:32.656ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9ac8dbf8-6117-4dbf-9a7b-d2328e810372", "content-length": "2", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4532024-05-30T04:30:32.656ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
4542024-05-30T04:30:32.656ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8a3ccb5b-16cc-4fac-b5cb-64e043b116ce", "content-length": "52", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4552024-05-30T04:30:32.656ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
4562024-05-30T04:30:32.656ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4572024-05-30T04:30:32.701ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3a6d31a4-25a3-4c0b-a651-44abc24ecfcb", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4582024-05-30T04:30:32.701ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
4592024-05-30T04:30:32.701ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4602024-05-30T04:30:32.701ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1c0edf73-add7-4715-b69b-e2cb4bb5f2f5", "content-length": "467", "date": "Thu, 30 May 2024 04:30:32 GMT"} })
4612024-05-30T04:30:32.701ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4622024-05-30T04:30:32.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4632024-05-30T04:30:32.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 160 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 17755613-3c99-4d55-b6ca-d5d5190f688d response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
4642024-05-30T04:30:32.704ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4652024-05-30T04:30:32.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4662024-05-30T04:30:32.707ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4672024-05-30T04:30:32.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4682024-05-30T04:30:32.708ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4692024-05-30T04:30:32.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4702024-05-30T04:30:32.714ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 552.534367ms iteration = 6
4712024-05-30T04:30:42.160ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
4722024-05-30T04:30:42.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4732024-05-30T04:30:42.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4742024-05-30T04:30:42.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4752024-05-30T04:30:42.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4762024-05-30T04:30:42.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4772024-05-30T04:30:42.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4782024-05-30T04:30:42.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4792024-05-30T04:30:42.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4802024-05-30T04:30:42.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4812024-05-30T04:30:42.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
4822024-05-30T04:30:42.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4832024-05-30T04:30:42.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4842024-05-30T04:30:42.175ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
4852024-05-30T04:30:42.175ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.456186ms iteration = 17
4862024-05-30T04:30:42.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 238 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = da044ae6-5106-40b7-95b1-b6c2d4880201 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
4872024-05-30T04:30:52.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
4882024-05-30T04:30:52.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4892024-05-30T04:30:52.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4902024-05-30T04:30:52.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4912024-05-30T04:30:52.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4922024-05-30T04:30:52.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4932024-05-30T04:30:52.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4942024-05-30T04:30:52.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4952024-05-30T04:30:52.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4962024-05-30T04:30:52.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4972024-05-30T04:30:52.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
4982024-05-30T04:30:52.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4992024-05-30T04:30:52.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5002024-05-30T04:30:52.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
5012024-05-30T04:30:52.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.327738ms iteration = 18
5022024-05-30T04:30:52.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 136 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 783affb2-c5de-4678-9fd4-334d72bd6c2d response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
5032024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
5042024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
5052024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
5062024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
5072024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
5082024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
5092024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
5102024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5112024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5122024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5132024-05-30T04:31:02.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5142024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5152024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5162024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5172024-05-30T04:31:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5182024-05-30T04:31:02.152ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 266486562 } })
5192024-05-30T04:31:02.152ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 266727205 } })
5202024-05-30T04:31:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5212024-05-30T04:31:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5222024-05-30T04:31:02.155ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
5232024-05-30T04:31:02.155ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.911067ms iteration = 4
5242024-05-30T04:31:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5252024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
5262024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 7.671769ms iteration = 4
5272024-05-30T04:31:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5282024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
5292024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5302024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5312024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
5322024-05-30T04:31:02.161ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
5332024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.67\u{b5}s iteration = 5
5342024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
5352024-05-30T04:31:02.161ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:21:02.161597536 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
5362024-05-30T04:31:02.161ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
5372024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
5382024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
5392024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5402024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
5412024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5422024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
5432024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5442024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5452024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
5462024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
5472024-05-30T04:31:02.162ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
5482024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
5492024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5502024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
5512024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5522024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
5532024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5542024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
5552024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5562024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
5572024-05-30T04:31:02.162ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
5582024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
5592024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
5602024-05-30T04:31:02.162ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
5612024-05-30T04:31:02.162ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
5622024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.83\u{b5}s iteration = 7
5632024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
5642024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5652024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5662024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5672024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
5682024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5692024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5702024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5712024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
5722024-05-30T04:31:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5732024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5742024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5752024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5762024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5772024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5782024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5792024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5802024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5812024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5822024-05-30T04:31:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5832024-05-30T04:31:02.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5842024-05-30T04:31:02.166ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5852024-05-30T04:31:02.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 280835387 } })
5862024-05-30T04:31:02.167ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
5872024-05-30T04:31:02.167ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5882024-05-30T04:31:02.167ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 281501835 } })
5892024-05-30T04:31:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5902024-05-30T04:31:02.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5912024-05-30T04:31:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5922024-05-30T04:31:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5932024-05-30T04:31:02.168ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5942024-05-30T04:31:02.168ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
5952024-05-30T04:31:02.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5962024-05-30T04:31:02.169ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5972024-05-30T04:31:02.169ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:21:02.161597536 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
5982024-05-30T04:31:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.995872ms iteration = 4
5992024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6002024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6012024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6022024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6032024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6042024-05-30T04:31:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6052024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6062024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6072024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6082024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6092024-05-30T04:31:02.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6102024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6112024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6122024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6132024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6142024-05-30T04:31:02.172ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
6152024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.440821ms iteration = 7
6162024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6172024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6182024-05-30T04:31:02.172ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
6192024-05-30T04:31:02.172ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
6202024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 10.715263ms iteration = 7
6212024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6222024-05-30T04:31:02.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6232024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6242024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6252024-05-30T04:31:02.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6262024-05-30T04:31:02.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6272024-05-30T04:31:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6282024-05-30T04:31:02.174ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1084, tv_nsec: 289222043 } })
6292024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6302024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6312024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6322024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6332024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6342024-05-30T04:31:02.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6352024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6362024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6372024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6382024-05-30T04:31:02.176ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
6392024-05-30T04:31:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6402024-05-30T04:31:02.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6412024-05-30T04:31:02.178ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
6422024-05-30T04:31:02.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6432024-05-30T04:31:02.178ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
6442024-05-30T04:31:02.178ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 16.008594ms iteration = 7
6452024-05-30T04:31:02.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6462024-05-30T04:31:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6472024-05-30T04:31:02.179ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6482024-05-30T04:31:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6492024-05-30T04:31:02.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6502024-05-30T04:31:02.181ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6512024-05-30T04:31:02.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6522024-05-30T04:31:02.183ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
6532024-05-30T04:31:02.183ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6542024-05-30T04:31:02.183ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
6552024-05-30T04:31:02.183ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 32.648706ms iteration = 4
6562024-05-30T04:31:02.190ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
6572024-05-30T04:31:02.190ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.809053ms iteration = 19
6582024-05-30T04:31:02.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6592024-05-30T04:31:02.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6602024-05-30T04:31:02.202ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6612024-05-30T04:31:02.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6622024-05-30T04:31:02.208ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6632024-05-30T04:31:02.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6642024-05-30T04:31:02.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6652024-05-30T04:31:02.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6662024-05-30T04:31:02.210ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6672024-05-30T04:31:02.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6682024-05-30T04:31:02.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692024-05-30T04:31:02.211ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6702024-05-30T04:31:02.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6712024-05-30T04:31:02.214ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6722024-05-30T04:31:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
6732024-05-30T04:31:02.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6742024-05-30T04:31:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6752024-05-30T04:31:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6762024-05-30T04:31:02.216ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 54.271774ms iteration = 7
6772024-05-30T04:31:02.224ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6782024-05-30T04:31:02.224ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
6792024-05-30T04:31:02.224ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6802024-05-30T04:31:02.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6812024-05-30T04:31:02.225ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6822024-05-30T04:31:02.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6832024-05-30T04:31:02.228ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6842024-05-30T04:31:02.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6852024-05-30T04:31:02.229ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
6862024-05-30T04:31:02.229ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6872024-05-30T04:31:02.229ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
6882024-05-30T04:31:02.229ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 79.237372ms iteration = 4
6892024-05-30T04:31:02.230ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
6902024-05-30T04:31:02.230ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
6912024-05-30T04:31:02.230ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 68.67453ms iteration = 4
6922024-05-30T04:31:02.234ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6932024-05-30T04:31:02.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6942024-05-30T04:31:02.235ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6952024-05-30T04:31:02.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6962024-05-30T04:31:02.236ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6972024-05-30T04:31:02.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6982024-05-30T04:31:02.238ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6992024-05-30T04:31:02.259ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7002024-05-30T04:31:02.285ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
7012024-05-30T04:31:02.285ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7022024-05-30T04:31:02.285ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
7032024-05-30T04:31:02.285ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 123.636022ms iteration = 4
7042024-05-30T04:31:02.285ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6cac0f24-7a2b-46db-941d-10bd7e907f7d", "content-length": "26", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7052024-05-30T04:31:02.285ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7062024-05-30T04:31:02.287ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
7072024-05-30T04:31:02.287ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7082024-05-30T04:31:02.287ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0eed8192-57c9-4e32-980e-509bfbfe748a", "content-length": "26", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7092024-05-30T04:31:02.287ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7102024-05-30T04:31:02.291ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7112024-05-30T04:31:02.291ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
7122024-05-30T04:31:02.291ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7132024-05-30T04:31:02.291ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f35cfd5-c1b6-46da-a357-0f11941abe20", "content-length": "26", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7142024-05-30T04:31:02.291ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7152024-05-30T04:31:02.291ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7162024-05-30T04:31:02.293ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
7172024-05-30T04:31:02.388ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7182024-05-30T04:31:02.424ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7192024-05-30T04:31:02.424ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "736686f2-790f-46f9-a31c-7035713cb2c8", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7202024-05-30T04:31:02.424ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7212024-05-30T04:31:02.425ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c033b170-80cc-4061-8caa-5b66a529bcdd", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7222024-05-30T04:31:02.425ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7232024-05-30T04:31:02.426ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7242024-05-30T04:31:02.426ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 264.485224ms iteration = 6
7252024-05-30T04:31:02.458ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 296.414031ms iteration = 7
7262024-05-30T04:31:02.458ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7272024-05-30T04:31:02.459ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ac8d787e-56de-42b4-aaa6-e3e327b5172f", "content-length": "1", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7282024-05-30T04:31:02.460ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7292024-05-30T04:31:02.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7302024-05-30T04:31:02.461ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 299.80533ms iteration = 7
7312024-05-30T04:31:02.513ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7322024-05-30T04:31:02.513ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b40cac34-8d99-450c-b7ff-c14e26a260e7", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7332024-05-30T04:31:02.513ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7342024-05-30T04:31:02.515ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9373a9e9-5e49-488c-8172-bed34e1ff732", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7352024-05-30T04:31:02.515ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7362024-05-30T04:31:02.517ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "99a7816d-ba48-4dcc-ac9f-c0f162766c28", "date": "Thu, 30 May 2024 04:31:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7372024-05-30T04:31:02.517ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7382024-05-30T04:31:02.518ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7392024-05-30T04:31:02.518ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 368.52249ms iteration = 6
7402024-05-30T04:31:02.531ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7412024-05-30T04:31:02.532ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "15a4bd6e-a15f-4e9a-826a-be011cd01a32", "content-length": "691", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7422024-05-30T04:31:02.532ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
7432024-05-30T04:31:02.532ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
7442024-05-30T04:31:02.532ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
7452024-05-30T04:31:02.533ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 371.414684ms iteration = 7
7462024-05-30T04:31:02.595ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7472024-05-30T04:31:02.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7482024-05-30T04:31:02.597ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
7492024-05-30T04:31:02.597ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7502024-05-30T04:31:02.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7512024-05-30T04:31:02.607ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7522024-05-30T04:31:02.607ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8c79283a-802e-45a7-b74c-9544612ee240", "content-length": "121", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7532024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
7542024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
7552024-05-30T04:31:02.607ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
7562024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
7572024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
7582024-05-30T04:31:02.607ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
7592024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
7602024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
7612024-05-30T04:31:02.607ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
7622024-05-30T04:31:02.607ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7632024-05-30T04:31:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7642024-05-30T04:31:02.609ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7652024-05-30T04:31:02.609ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "492c4824-b717-456d-ae95-2cefccdf7a61", "content-length": "2", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7662024-05-30T04:31:02.609ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7672024-05-30T04:31:02.610ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9250780a-d3b1-4b37-9368-7b28105296d1", "content-length": "52", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7682024-05-30T04:31:02.610ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
7692024-05-30T04:31:02.610ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7702024-05-30T04:31:02.658ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1c1e64fd-6857-45c3-8772-670f3ae58eaa", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7712024-05-30T04:31:02.658ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
7722024-05-30T04:31:02.658ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7732024-05-30T04:31:02.658ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "28daed6e-1b9f-4c1a-9709-391a824f5382", "content-length": "467", "date": "Thu, 30 May 2024 04:31:02 GMT"} })
7742024-05-30T04:31:02.658ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7752024-05-30T04:31:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7762024-05-30T04:31:02.662ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7772024-05-30T04:31:02.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7782024-05-30T04:31:02.665ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7792024-05-30T04:31:02.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7802024-05-30T04:31:02.666ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7812024-05-30T04:31:02.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7822024-05-30T04:31:02.674ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 512.001401ms iteration = 7
7832024-05-30T04:31:02.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 141 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 792772bd-181a-4449-b91e-ccb02b64efa7 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
7842024-05-30T04:31:12.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
7852024-05-30T04:31:12.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7862024-05-30T04:31:12.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7872024-05-30T04:31:12.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7882024-05-30T04:31:12.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7892024-05-30T04:31:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7902024-05-30T04:31:12.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7912024-05-30T04:31:12.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7922024-05-30T04:31:12.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7932024-05-30T04:31:12.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942024-05-30T04:31:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
7952024-05-30T04:31:12.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7962024-05-30T04:31:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7972024-05-30T04:31:12.176ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
7982024-05-30T04:31:12.176ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.952842ms iteration = 20
7992024-05-30T04:31:12.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 154 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 83571564-9b9e-4a70-8c5e-9d68f5d6b548 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
8002024-05-30T04:31:22.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
8012024-05-30T04:31:22.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8022024-05-30T04:31:22.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8032024-05-30T04:31:22.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8042024-05-30T04:31:22.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8052024-05-30T04:31:22.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8062024-05-30T04:31:22.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8072024-05-30T04:31:22.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8082024-05-30T04:31:22.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8092024-05-30T04:31:22.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8102024-05-30T04:31:22.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
8112024-05-30T04:31:22.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8122024-05-30T04:31:22.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8132024-05-30T04:31:22.176ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
8142024-05-30T04:31:22.176ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.007953ms iteration = 21
8152024-05-30T04:31:22.704ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 160 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 470c3928-c842-4dcc-b6a1-f46b29ce6b00 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
8162024-05-30T04:31:32.161ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
8172024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
8182024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8192024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
8202024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8212024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8222024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
8232024-05-30T04:31:32.162ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8242024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8252024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8262024-05-30T04:31:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8272024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
8282024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
8292024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
8302024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8312024-05-30T04:31:32.163ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
8322024-05-30T04:31:32.163ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
8332024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8342024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.43\u{b5}s iteration = 8
8352024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
8362024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
8372024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8382024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
8392024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8402024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8412024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8422024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8432024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8442024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8452024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8462024-05-30T04:31:32.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8472024-05-30T04:31:32.164ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1114, tv_nsec: 277799032 } })
8482024-05-30T04:31:32.164ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1114, tv_nsec: 278024225 } })
8492024-05-30T04:31:32.164ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8502024-05-30T04:31:32.164ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.769462ms iteration = 8
8512024-05-30T04:31:32.164ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8522024-05-30T04:31:32.165ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8532024-05-30T04:31:32.165ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
8542024-05-30T04:31:32.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8552024-05-30T04:31:32.166ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
8562024-05-30T04:31:32.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8572024-05-30T04:31:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8582024-05-30T04:31:32.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8592024-05-30T04:31:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8602024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8612024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8622024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8632024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8642024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8652024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8662024-05-30T04:31:32.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8672024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8682024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8692024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8702024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8712024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8722024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8732024-05-30T04:31:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8742024-05-30T04:31:32.170ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
8752024-05-30T04:31:32.170ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
8762024-05-30T04:31:32.170ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 6.724397ms iteration = 8
8772024-05-30T04:31:32.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8782024-05-30T04:31:32.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8792024-05-30T04:31:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8802024-05-30T04:31:32.171ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1114, tv_nsec: 285042925 } })
8812024-05-30T04:31:32.171ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
8822024-05-30T04:31:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
8832024-05-30T04:31:32.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8842024-05-30T04:31:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8852024-05-30T04:31:32.178ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
8862024-05-30T04:31:32.178ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.678482ms iteration = 22
8872024-05-30T04:31:32.186ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8882024-05-30T04:31:32.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8892024-05-30T04:31:32.192ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8902024-05-30T04:31:32.193ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8912024-05-30T04:31:32.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8922024-05-30T04:31:32.194ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
8932024-05-30T04:31:32.194ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
8942024-05-30T04:31:32.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
8952024-05-30T04:31:32.195ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8962024-05-30T04:31:32.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8972024-05-30T04:31:32.195ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8982024-05-30T04:31:32.196ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8992024-05-30T04:31:32.198ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9002024-05-30T04:31:32.198ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9012024-05-30T04:31:32.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9022024-05-30T04:31:32.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
9032024-05-30T04:31:32.198ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9042024-05-30T04:31:32.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9052024-05-30T04:31:32.200ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 37.266458ms iteration = 8
9062024-05-30T04:31:32.236ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
9072024-05-30T04:31:32.236ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 72.978689ms iteration = 8
9082024-05-30T04:31:32.246ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
9092024-05-30T04:31:32.246ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9102024-05-30T04:31:32.246ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6b3a48a2-4b9d-4fa8-9fab-bd48c1f9ed3f", "content-length": "26", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9112024-05-30T04:31:32.246ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9122024-05-30T04:31:32.247ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9132024-05-30T04:31:32.256ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
9142024-05-30T04:31:32.256ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9152024-05-30T04:31:32.257ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a2e28b2-fd79-4195-bf21-11d999df04c8", "content-length": "26", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9162024-05-30T04:31:32.257ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9172024-05-30T04:31:32.257ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
9182024-05-30T04:31:32.257ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9192024-05-30T04:31:32.257ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "00007ce6-f43a-41c9-8ba2-62adc493b8da", "content-length": "26", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9202024-05-30T04:31:32.257ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9212024-05-30T04:31:32.259ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9222024-05-30T04:31:32.260ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
9232024-05-30T04:31:32.295ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
9242024-05-30T04:31:32.295ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8097cacb-652b-4c2d-b1f2-819b13c31d3c", "content-length": "691", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9252024-05-30T04:31:32.295ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
9262024-05-30T04:31:32.295ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
9272024-05-30T04:31:32.295ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
9282024-05-30T04:31:32.296ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 133.144872ms iteration = 8
9292024-05-30T04:31:32.342ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9302024-05-30T04:31:32.342ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "760f8e8c-2080-453d-a137-0136e4bcbcac", "content-length": "1", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9312024-05-30T04:31:32.343ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9322024-05-30T04:31:32.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9332024-05-30T04:31:32.345ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 183.285569ms iteration = 8
9342024-05-30T04:31:32.355ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 193.084672ms iteration = 8
9352024-05-30T04:31:32.414ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9362024-05-30T04:31:32.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9372024-05-30T04:31:32.416ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
9382024-05-30T04:31:32.416ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9392024-05-30T04:31:32.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9402024-05-30T04:31:32.422ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9412024-05-30T04:31:32.422ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ee46baf8-de22-4868-a8cd-394f6c118dec", "content-length": "121", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9422024-05-30T04:31:32.422ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
9432024-05-30T04:31:32.422ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
9442024-05-30T04:31:32.422ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
9452024-05-30T04:31:32.423ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
9462024-05-30T04:31:32.423ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
9472024-05-30T04:31:32.423ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
9482024-05-30T04:31:32.423ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
9492024-05-30T04:31:32.423ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
9502024-05-30T04:31:32.423ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
9512024-05-30T04:31:32.423ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9522024-05-30T04:31:32.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9532024-05-30T04:31:32.424ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9542024-05-30T04:31:32.424ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ffd67e4f-6527-4802-874f-e0e6c7e866d3", "content-length": "2", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9552024-05-30T04:31:32.425ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9562024-05-30T04:31:32.425ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3cdde998-30df-49a3-8012-ffc970af6b5e", "content-length": "52", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9572024-05-30T04:31:32.425ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
9582024-05-30T04:31:32.425ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9592024-05-30T04:31:32.516ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7d9098a1-654d-4489-8b74-ccbd4f947b7e", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9602024-05-30T04:31:32.516ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
9612024-05-30T04:31:32.516ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9622024-05-30T04:31:32.521ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ab1c93a5-bfa2-4226-bc04-39b965c8c08a", "content-length": "467", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9632024-05-30T04:31:32.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9642024-05-30T04:31:32.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9652024-05-30T04:31:32.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9662024-05-30T04:31:32.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9672024-05-30T04:31:32.527ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9682024-05-30T04:31:32.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9692024-05-30T04:31:32.528ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9702024-05-30T04:31:32.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9712024-05-30T04:31:32.535ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 372.282954ms iteration = 8
9722024-05-30T04:31:32.676ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
9732024-05-30T04:31:32.676ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
9742024-05-30T04:31:32.706ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
9752024-05-30T04:31:32.706ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:51735
9762024-05-30T04:31:32.706ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 133 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 49435f28-fd73-49dc-9109-0593348d2207 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
9772024-05-30T04:31:32.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51735 req_id = cf2a0683-dc81-4bd5-9234-a6ef8fcaf9b7 roles = RoleSet { roles: {} } uri = /metrics/producers
9782024-05-30T04:31:32.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51735 req_id = cf2a0683-dc81-4bd5-9234-a6ef8fcaf9b7 resource = Database result = Ok(()) uri = /metrics/producers
9792024-05-30T04:31:32.737ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
9802024-05-30T04:31:32.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51735 req_id = cf2a0683-dc81-4bd5-9234-a6ef8fcaf9b7 roles = RoleSet { roles: {} } uri = /metrics/producers
9812024-05-30T04:31:32.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51735 req_id = cf2a0683-dc81-4bd5-9234-a6ef8fcaf9b7 resource = Database result = Ok(()) uri = /metrics/producers
9822024-05-30T04:31:32.769ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
9832024-05-30T04:31:32.770ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "48b12a2c-953a-420e-b738-c6a76e25375c", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9842024-05-30T04:31:32.770ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = b5bf7e90-33d3-4f67-b9d3-793143654e9a
9852024-05-30T04:31:32.771ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 65302 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:51735 req_id = cf2a0683-dc81-4bd5-9234-a6ef8fcaf9b7 response_code = 201 uri = /metrics/producers
9862024-05-30T04:31:32.771ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "cf2a0683-dc81-4bd5-9234-a6ef8fcaf9b7", "content-length": "41", "date": "Thu, 30 May 2024 04:31:32 GMT"} })
9872024-05-30T04:31:32.773ZDEBGnexus (producer-registration-task): registered with nexus successfully
9882024-05-30T04:31:32.773ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
9892024-05-30T04:31:42.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
9902024-05-30T04:31:42.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9912024-05-30T04:31:42.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9922024-05-30T04:31:42.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9932024-05-30T04:31:42.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9942024-05-30T04:31:42.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9952024-05-30T04:31:42.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9962024-05-30T04:31:42.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9972024-05-30T04:31:42.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9982024-05-30T04:31:42.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9992024-05-30T04:31:42.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
10002024-05-30T04:31:42.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10012024-05-30T04:31:42.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10022024-05-30T04:31:42.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
10032024-05-30T04:31:42.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.525977ms iteration = 23
10042024-05-30T04:31:42.770ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 148 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 718dab4a-e3c2-4f15-93d7-6e7132f3556e response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
10052024-05-30T04:31:52.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
10062024-05-30T04:31:52.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10072024-05-30T04:31:52.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10082024-05-30T04:31:52.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10092024-05-30T04:31:52.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10102024-05-30T04:31:52.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10112024-05-30T04:31:52.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10122024-05-30T04:31:52.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10132024-05-30T04:31:52.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10142024-05-30T04:31:52.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10152024-05-30T04:31:52.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
10162024-05-30T04:31:52.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10172024-05-30T04:31:52.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10182024-05-30T04:31:52.176ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
10192024-05-30T04:31:52.176ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.293495ms iteration = 24
10202024-05-30T04:31:52.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 249 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 1eb8d53e-28a4-47f3-b051-2d7cdff64051 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
10212024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
10222024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
10232024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
10242024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
10252024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10262024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10272024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
10282024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10292024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10302024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10312024-05-30T04:32:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10322024-05-30T04:32:02.151ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 265005095 } })
10332024-05-30T04:32:02.151ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 265048476 } })
10342024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10352024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10362024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10372024-05-30T04:32:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10382024-05-30T04:32:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10392024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10402024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10412024-05-30T04:32:02.154ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.49855ms iteration = 5
10422024-05-30T04:32:02.156ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
10432024-05-30T04:32:02.156ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.009848ms iteration = 5
10442024-05-30T04:32:02.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10452024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10462024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10472024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10482024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10492024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10502024-05-30T04:32:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10512024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10522024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10532024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10542024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10552024-05-30T04:32:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10562024-05-30T04:32:02.160ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10572024-05-30T04:32:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10582024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
10592024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
10602024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10612024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
10622024-05-30T04:32:02.162ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:22:02.162620837 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
10632024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
10642024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
10652024-05-30T04:32:02.162ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
10662024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.46\u{b5}s iteration = 6
10672024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
10682024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
10692024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10702024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
10712024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
10722024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10732024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
10742024-05-30T04:32:02.162ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
10752024-05-30T04:32:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
10762024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10772024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10782024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10792024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
10802024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10812024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10822024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10832024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
10842024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
10852024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10862024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
10872024-05-30T04:32:02.163ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
10882024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
10892024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10902024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
10912024-05-30T04:32:02.163ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
10922024-05-30T04:32:02.163ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
10932024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.06\u{b5}s iteration = 9
10942024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
10952024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
10962024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10972024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10982024-05-30T04:32:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10992024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
11002024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
11012024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11022024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11032024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11042024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11052024-05-30T04:32:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11062024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11072024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11082024-05-30T04:32:02.165ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:22:02.162620837 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
11092024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.818732ms iteration = 5
11102024-05-30T04:32:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11112024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11122024-05-30T04:32:02.166ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
11132024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.560851ms iteration = 9
11142024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11152024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
11162024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
11172024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11182024-05-30T04:32:02.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 279886996 } })
11192024-05-30T04:32:02.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 279909427 } })
11202024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11212024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11222024-05-30T04:32:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11232024-05-30T04:32:02.167ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
11242024-05-30T04:32:02.167ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11252024-05-30T04:32:02.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11262024-05-30T04:32:02.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11272024-05-30T04:32:02.168ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
11282024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11292024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11302024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11312024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11322024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11332024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11342024-05-30T04:32:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11352024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11362024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11372024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11382024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11392024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11402024-05-30T04:32:02.170ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11412024-05-30T04:32:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11422024-05-30T04:32:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11432024-05-30T04:32:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11442024-05-30T04:32:02.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11452024-05-30T04:32:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11462024-05-30T04:32:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11472024-05-30T04:32:02.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11482024-05-30T04:32:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11492024-05-30T04:32:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11502024-05-30T04:32:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11512024-05-30T04:32:02.176ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1144, tv_nsec: 289398016 } })
11522024-05-30T04:32:02.176ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
11532024-05-30T04:32:02.179ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
11542024-05-30T04:32:02.179ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11552024-05-30T04:32:02.179ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
11562024-05-30T04:32:02.179ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 28.046923ms iteration = 5
11572024-05-30T04:32:02.183ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
11582024-05-30T04:32:02.183ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
11592024-05-30T04:32:02.183ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.640536ms iteration = 9
11602024-05-30T04:32:02.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11612024-05-30T04:32:02.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11622024-05-30T04:32:02.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11632024-05-30T04:32:02.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11642024-05-30T04:32:02.186ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11652024-05-30T04:32:02.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11662024-05-30T04:32:02.188ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
11672024-05-30T04:32:02.188ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
11682024-05-30T04:32:02.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11692024-05-30T04:32:02.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11702024-05-30T04:32:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
11712024-05-30T04:32:02.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11722024-05-30T04:32:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11732024-05-30T04:32:02.190ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11742024-05-30T04:32:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
11752024-05-30T04:32:02.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11762024-05-30T04:32:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11772024-05-30T04:32:02.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11782024-05-30T04:32:02.192ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11792024-05-30T04:32:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11802024-05-30T04:32:02.193ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11812024-05-30T04:32:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11822024-05-30T04:32:02.194ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11832024-05-30T04:32:02.195ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
11842024-05-30T04:32:02.195ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
11852024-05-30T04:32:02.195ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
11862024-05-30T04:32:02.195ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 44.31791ms iteration = 5
11872024-05-30T04:32:02.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11882024-05-30T04:32:02.196ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11892024-05-30T04:32:02.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11902024-05-30T04:32:02.197ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
11912024-05-30T04:32:02.197ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.80928ms iteration = 25
11922024-05-30T04:32:02.197ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11932024-05-30T04:32:02.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11942024-05-30T04:32:02.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11952024-05-30T04:32:02.199ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11962024-05-30T04:32:02.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11972024-05-30T04:32:02.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11982024-05-30T04:32:02.200ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11992024-05-30T04:32:02.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12002024-05-30T04:32:02.201ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12012024-05-30T04:32:02.201ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12022024-05-30T04:32:02.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12032024-05-30T04:32:02.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12042024-05-30T04:32:02.203ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12052024-05-30T04:32:02.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12062024-05-30T04:32:02.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12072024-05-30T04:32:02.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
12082024-05-30T04:32:02.205ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12092024-05-30T04:32:02.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12102024-05-30T04:32:02.207ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 43.41792ms iteration = 9
12112024-05-30T04:32:02.231ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
12122024-05-30T04:32:02.231ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
12132024-05-30T04:32:02.231ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 67.605748ms iteration = 5
12142024-05-30T04:32:02.245ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
12152024-05-30T04:32:02.245ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 82.746492ms iteration = 5
12162024-05-30T04:32:02.265ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
12172024-05-30T04:32:02.265ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 101.294025ms iteration = 9
12182024-05-30T04:32:02.344ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
12192024-05-30T04:32:02.344ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12202024-05-30T04:32:02.346ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "be589d6f-9677-4881-8bd4-764bb0672ef0", "content-length": "26", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12212024-05-30T04:32:02.346ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12222024-05-30T04:32:02.347ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12232024-05-30T04:32:02.371ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12242024-05-30T04:32:02.373ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12252024-05-30T04:32:02.374ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a495410f-a0a1-49df-8f4d-62e77e449de3", "content-length": "691", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12262024-05-30T04:32:02.374ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
12272024-05-30T04:32:02.374ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
12282024-05-30T04:32:02.374ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
12292024-05-30T04:32:02.375ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 211.896448ms iteration = 9
12302024-05-30T04:32:02.466ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12312024-05-30T04:32:02.478ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
12322024-05-30T04:32:02.479ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12332024-05-30T04:32:02.479ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "06545427-22f2-4732-8206-825ebdff7630", "content-length": "26", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12342024-05-30T04:32:02.479ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12352024-05-30T04:32:02.481ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12362024-05-30T04:32:02.491ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
12372024-05-30T04:32:02.491ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12382024-05-30T04:32:02.491ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e995730b-ba39-460b-9700-2973996502bd", "content-length": "26", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12392024-05-30T04:32:02.491ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12402024-05-30T04:32:02.493ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
12412024-05-30T04:32:02.496ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12422024-05-30T04:32:02.497ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "65135c1c-1189-4514-b2dc-fec24d0fe038", "content-length": "1", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12432024-05-30T04:32:02.498ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12442024-05-30T04:32:02.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12452024-05-30T04:32:02.500ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 338.0824ms iteration = 9
12462024-05-30T04:32:02.547ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
12472024-05-30T04:32:02.576ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
12482024-05-30T04:32:02.577ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "47648b96-bb96-42fe-aac4-b76d38ca1b91", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
12492024-05-30T04:32:02.577ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
12502024-05-30T04:32:02.579ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a6f8eddd-75dc-40df-8c22-4ab9d2a4f12b", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
12512024-05-30T04:32:02.579ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
12522024-05-30T04:32:02.580ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
12532024-05-30T04:32:02.580ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 418.298683ms iteration = 7
12542024-05-30T04:32:02.607ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 444.359412ms iteration = 9
12552024-05-30T04:32:02.613ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12562024-05-30T04:32:02.613ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "38d16564-d0f7-495e-8d4c-6c70ce87107e", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12572024-05-30T04:32:02.613ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12582024-05-30T04:32:02.616ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "51804f81-6655-4b0a-b640-cfbf5851ddba", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12592024-05-30T04:32:02.616ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12602024-05-30T04:32:02.617ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1e1133bd-7fa2-4ac6-b2fe-f51f97519f43", "date": "Thu, 30 May 2024 04:32:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12612024-05-30T04:32:02.617ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12622024-05-30T04:32:02.619ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12632024-05-30T04:32:02.619ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 468.228158ms iteration = 7
12642024-05-30T04:32:02.670ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12652024-05-30T04:32:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12662024-05-30T04:32:02.672ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
12672024-05-30T04:32:02.672ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12682024-05-30T04:32:02.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12692024-05-30T04:32:02.681ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
12702024-05-30T04:32:02.682ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6fb45cd8-e47a-4203-a26f-ad328a85c590", "content-length": "121", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12712024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
12722024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
12732024-05-30T04:32:02.682ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
12742024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
12752024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
12762024-05-30T04:32:02.682ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
12772024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
12782024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
12792024-05-30T04:32:02.682ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
12802024-05-30T04:32:02.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12812024-05-30T04:32:02.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12822024-05-30T04:32:02.683ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
12832024-05-30T04:32:02.684ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "149165ea-8549-4248-98f5-4a2c4d22fdcc", "content-length": "2", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12842024-05-30T04:32:02.684ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
12852024-05-30T04:32:02.685ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c047dbc6-8a8c-4a2b-ba1f-dea992a53101", "content-length": "52", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12862024-05-30T04:32:02.685ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
12872024-05-30T04:32:02.685ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12882024-05-30T04:32:02.729ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "38a0a494-52e3-4614-82bb-e0dd0224b33e", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12892024-05-30T04:32:02.729ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
12902024-05-30T04:32:02.729ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12912024-05-30T04:32:02.730ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c7001e22-9a76-4adb-96e6-138346633ac5", "content-length": "467", "date": "Thu, 30 May 2024 04:32:02 GMT"} })
12922024-05-30T04:32:02.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12932024-05-30T04:32:02.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12942024-05-30T04:32:02.733ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12952024-05-30T04:32:02.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12962024-05-30T04:32:02.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12972024-05-30T04:32:02.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12982024-05-30T04:32:02.736ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12992024-05-30T04:32:02.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13002024-05-30T04:32:02.743ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 580.255087ms iteration = 9
13012024-05-30T04:32:02.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 752 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = d9b479c3-2be3-4149-9063-76b240d846d2 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
13022024-05-30T04:32:12.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
13032024-05-30T04:32:12.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13042024-05-30T04:32:12.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13052024-05-30T04:32:12.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062024-05-30T04:32:12.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13072024-05-30T04:32:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13082024-05-30T04:32:12.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13092024-05-30T04:32:12.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13102024-05-30T04:32:12.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13112024-05-30T04:32:12.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13122024-05-30T04:32:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
13132024-05-30T04:32:12.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13142024-05-30T04:32:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13152024-05-30T04:32:12.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
13162024-05-30T04:32:12.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.72018ms iteration = 26
13172024-05-30T04:32:12.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 140 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 63cad4fe-85a1-4e6d-a7e8-7d616375c9ff response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
13182024-05-30T04:32:22.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
13192024-05-30T04:32:22.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13202024-05-30T04:32:22.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13212024-05-30T04:32:22.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13222024-05-30T04:32:22.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13232024-05-30T04:32:22.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13242024-05-30T04:32:22.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13252024-05-30T04:32:22.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13262024-05-30T04:32:22.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13272024-05-30T04:32:22.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13282024-05-30T04:32:22.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
13292024-05-30T04:32:22.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13302024-05-30T04:32:22.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13312024-05-30T04:32:22.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
13322024-05-30T04:32:22.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.502747ms iteration = 27
13332024-05-30T04:32:22.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 137 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = f84d8a80-4eb4-40d0-97b9-d4eb03f365c4 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
13342024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
13352024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13362024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
13372024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13382024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
13392024-05-30T04:32:32.162ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
13402024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
13412024-05-30T04:32:32.162ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13422024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13432024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13442024-05-30T04:32:32.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13452024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
13462024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
13472024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13482024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
13492024-05-30T04:32:32.164ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
13502024-05-30T04:32:32.164ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
13512024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13522024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.59\u{b5}s iteration = 10
13532024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
13542024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
13552024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13562024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13572024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
13582024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13592024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13602024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13612024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13622024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13632024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13642024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13652024-05-30T04:32:32.164ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
13662024-05-30T04:32:32.164ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.048954ms iteration = 10
13672024-05-30T04:32:32.165ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1174, tv_nsec: 277928354 } })
13682024-05-30T04:32:32.165ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13692024-05-30T04:32:32.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13702024-05-30T04:32:32.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13712024-05-30T04:32:32.166ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
13722024-05-30T04:32:32.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1174, tv_nsec: 279532602 } })
13732024-05-30T04:32:32.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742024-05-30T04:32:32.167ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
13752024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13762024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13772024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13782024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13792024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13802024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13812024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13822024-05-30T04:32:32.168ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13832024-05-30T04:32:32.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13842024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13852024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13862024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13872024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13882024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13892024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13902024-05-30T04:32:32.170ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1174, tv_nsec: 283595789 } })
13912024-05-30T04:32:32.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13922024-05-30T04:32:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13932024-05-30T04:32:32.171ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
13942024-05-30T04:32:32.171ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13952024-05-30T04:32:32.171ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
13962024-05-30T04:32:32.171ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
13972024-05-30T04:32:32.171ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 7.555077ms iteration = 10
13982024-05-30T04:32:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13992024-05-30T04:32:32.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14002024-05-30T04:32:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
14012024-05-30T04:32:32.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14022024-05-30T04:32:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14032024-05-30T04:32:32.182ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
14042024-05-30T04:32:32.182ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.614934ms iteration = 10
14052024-05-30T04:32:32.189ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
14062024-05-30T04:32:32.189ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.614986ms iteration = 28
14072024-05-30T04:32:32.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14082024-05-30T04:32:32.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14092024-05-30T04:32:32.192ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14102024-05-30T04:32:32.193ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14112024-05-30T04:32:32.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14122024-05-30T04:32:32.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14132024-05-30T04:32:32.195ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14142024-05-30T04:32:32.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14152024-05-30T04:32:32.196ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14162024-05-30T04:32:32.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14172024-05-30T04:32:32.198ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14182024-05-30T04:32:32.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14192024-05-30T04:32:32.200ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14202024-05-30T04:32:32.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14212024-05-30T04:32:32.201ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14222024-05-30T04:32:32.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
14232024-05-30T04:32:32.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14242024-05-30T04:32:32.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14252024-05-30T04:32:32.205ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 40.86808ms iteration = 10
14262024-05-30T04:32:32.263ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
14272024-05-30T04:32:32.263ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14282024-05-30T04:32:32.263ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e190a7cd-4cbc-4673-941e-18ac4caeb168", "content-length": "26", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14292024-05-30T04:32:32.263ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14302024-05-30T04:32:32.265ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14312024-05-30T04:32:32.321ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
14322024-05-30T04:32:32.321ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "97671343-d27c-4678-838f-24eb75e4fa7e", "content-length": "691", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14332024-05-30T04:32:32.321ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
14342024-05-30T04:32:32.321ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
14352024-05-30T04:32:32.321ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
14362024-05-30T04:32:32.322ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 158.545505ms iteration = 10
14372024-05-30T04:32:32.374ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
14382024-05-30T04:32:32.374ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14392024-05-30T04:32:32.374ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5dbb174a-cf04-4b90-b239-c1ff76e2e0b1", "content-length": "26", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14402024-05-30T04:32:32.374ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14412024-05-30T04:32:32.377ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14422024-05-30T04:32:32.378ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 215.467319ms iteration = 10
14432024-05-30T04:32:32.387ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
14442024-05-30T04:32:32.387ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14452024-05-30T04:32:32.387ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9e242494-cf7a-4eac-9c56-d05c389896c6", "content-length": "26", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14462024-05-30T04:32:32.387ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14472024-05-30T04:32:32.388ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
14482024-05-30T04:32:32.416ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14492024-05-30T04:32:32.416ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f18a31a7-f3de-4e45-898a-7bb54dac240a", "content-length": "1", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14502024-05-30T04:32:32.418ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14512024-05-30T04:32:32.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14522024-05-30T04:32:32.419ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 256.895636ms iteration = 10
14532024-05-30T04:32:32.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14542024-05-30T04:32:32.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14552024-05-30T04:32:32.485ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
14562024-05-30T04:32:32.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14572024-05-30T04:32:32.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14582024-05-30T04:32:32.491ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
14592024-05-30T04:32:32.491ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "008b5882-2e70-45cb-ab17-b687aec7f721", "content-length": "121", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14602024-05-30T04:32:32.491ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
14612024-05-30T04:32:32.491ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
14622024-05-30T04:32:32.492ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
14632024-05-30T04:32:32.492ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
14642024-05-30T04:32:32.492ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
14652024-05-30T04:32:32.492ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
14662024-05-30T04:32:32.492ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
14672024-05-30T04:32:32.492ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
14682024-05-30T04:32:32.492ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
14692024-05-30T04:32:32.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14702024-05-30T04:32:32.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14712024-05-30T04:32:32.493ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
14722024-05-30T04:32:32.493ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "29110d30-00f5-41b8-b7d8-11e800084975", "content-length": "2", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14732024-05-30T04:32:32.493ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
14742024-05-30T04:32:32.493ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a05f215e-58ab-4970-b92c-b5fbf1b15abc", "content-length": "52", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14752024-05-30T04:32:32.493ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
14762024-05-30T04:32:32.493ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14772024-05-30T04:32:32.539ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "2d472612-5b8d-4311-914a-0899d82f902c", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14782024-05-30T04:32:32.539ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
14792024-05-30T04:32:32.539ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14802024-05-30T04:32:32.539ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a73f16c-a2b4-449c-8dc0-1f495c6e97a9", "content-length": "467", "date": "Thu, 30 May 2024 04:32:32 GMT"} })
14812024-05-30T04:32:32.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14822024-05-30T04:32:32.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14832024-05-30T04:32:32.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14842024-05-30T04:32:32.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14852024-05-30T04:32:32.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14862024-05-30T04:32:32.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14872024-05-30T04:32:32.546ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14882024-05-30T04:32:32.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14892024-05-30T04:32:32.552ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 387.872782ms iteration = 10
14902024-05-30T04:32:32.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 165 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 6294a865-bfab-493d-b2c5-8f27d631e661 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
14912024-05-30T04:32:42.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
14922024-05-30T04:32:42.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14932024-05-30T04:32:42.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14942024-05-30T04:32:42.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14952024-05-30T04:32:42.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14962024-05-30T04:32:42.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14972024-05-30T04:32:42.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14982024-05-30T04:32:42.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14992024-05-30T04:32:42.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15002024-05-30T04:32:42.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15012024-05-30T04:32:42.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
15022024-05-30T04:32:42.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15032024-05-30T04:32:42.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15042024-05-30T04:32:42.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
15052024-05-30T04:32:42.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.582898ms iteration = 29
15062024-05-30T04:32:42.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 170 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = e5ea5a5f-ef9f-4a70-ab24-41f62d0a329c response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
15072024-05-30T04:32:52.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
15082024-05-30T04:32:52.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15092024-05-30T04:32:52.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15102024-05-30T04:32:52.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15112024-05-30T04:32:52.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15122024-05-30T04:32:52.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15132024-05-30T04:32:52.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15142024-05-30T04:32:52.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15152024-05-30T04:32:52.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15162024-05-30T04:32:52.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15172024-05-30T04:32:52.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
15182024-05-30T04:32:52.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15192024-05-30T04:32:52.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15202024-05-30T04:32:52.177ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
15212024-05-30T04:32:52.177ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.096073ms iteration = 30
15222024-05-30T04:32:52.772ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 230 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = e249a827-6910-4dcf-948b-e5e01be97e8b response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
15232024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
15242024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
15252024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
15262024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
15272024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
15282024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
15292024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
15302024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15312024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15322024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15332024-05-30T04:33:02.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15342024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15352024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15362024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15372024-05-30T04:33:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15382024-05-30T04:33:02.153ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 266127888 } })
15392024-05-30T04:33:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15402024-05-30T04:33:02.155ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 267372892 } })
15412024-05-30T04:33:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15422024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
15432024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 6.09139ms iteration = 6
15442024-05-30T04:33:02.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15452024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15462024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15472024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15482024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15492024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
15502024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.549107ms iteration = 6
15512024-05-30T04:33:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15522024-05-30T04:33:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15532024-05-30T04:33:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15542024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
15552024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
15562024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
15572024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15582024-05-30T04:33:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15592024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
15602024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
15612024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
15622024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15632024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
15642024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15652024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15662024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
15672024-05-30T04:33:02.163ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
15682024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
15692024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
15702024-05-30T04:33:02.163ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:23:02.163140542 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
15712024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15722024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
15732024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
15742024-05-30T04:33:02.163ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
15752024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.66\u{b5}s iteration = 7
15762024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15772024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15782024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15792024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
15802024-05-30T04:33:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
15812024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15822024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
15832024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
15842024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15852024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15862024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
15872024-05-30T04:33:02.164ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
15882024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
15892024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
15902024-05-30T04:33:02.164ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
15912024-05-30T04:33:02.164ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
15922024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.551\u{b5}s iteration = 11
15932024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
15942024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
15952024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15962024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
15972024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15982024-05-30T04:33:02.164ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
15992024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16002024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
16012024-05-30T04:33:02.164ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16022024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16032024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
16042024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16052024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16062024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16072024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16082024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16092024-05-30T04:33:02.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16102024-05-30T04:33:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16112024-05-30T04:33:02.166ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16122024-05-30T04:33:02.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16132024-05-30T04:33:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16142024-05-30T04:33:02.166ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:23:02.163140542 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
16152024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.854394ms iteration = 6
16162024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16172024-05-30T04:33:02.167ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
16182024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.087797ms iteration = 11
16192024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16202024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16212024-05-30T04:33:02.167ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16222024-05-30T04:33:02.168ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16232024-05-30T04:33:02.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 280543564 } })
16242024-05-30T04:33:02.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 280735766 } })
16252024-05-30T04:33:02.168ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16262024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16272024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
16282024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
16292024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16302024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16312024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16322024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16332024-05-30T04:33:02.169ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16342024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16352024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16362024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16372024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16382024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
16392024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16402024-05-30T04:33:02.170ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
16412024-05-30T04:33:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16422024-05-30T04:33:02.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16432024-05-30T04:33:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16442024-05-30T04:33:02.171ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16452024-05-30T04:33:02.171ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16462024-05-30T04:33:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16472024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16482024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16492024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16502024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16512024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
16522024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
16532024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16542024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
16552024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16562024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16572024-05-30T04:33:02.172ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
16582024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16592024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16602024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16612024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16622024-05-30T04:33:02.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16632024-05-30T04:33:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16642024-05-30T04:33:02.173ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16652024-05-30T04:33:02.174ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
16662024-05-30T04:33:02.174ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
16672024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 9.53214ms iteration = 11
16682024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16692024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16702024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16712024-05-30T04:33:02.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16722024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16732024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16742024-05-30T04:33:02.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1204, tv_nsec: 287794427 } })
16752024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16762024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16772024-05-30T04:33:02.175ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
16782024-05-30T04:33:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16792024-05-30T04:33:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16802024-05-30T04:33:02.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16812024-05-30T04:33:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16822024-05-30T04:33:02.177ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16832024-05-30T04:33:02.177ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16842024-05-30T04:33:02.178ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
16852024-05-30T04:33:02.178ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
16862024-05-30T04:33:02.178ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
16872024-05-30T04:33:02.178ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 26.422544ms iteration = 6
16882024-05-30T04:33:02.178ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
16892024-05-30T04:33:02.178ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.118173ms iteration = 11
16902024-05-30T04:33:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
16912024-05-30T04:33:02.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16922024-05-30T04:33:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16932024-05-30T04:33:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16942024-05-30T04:33:02.180ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16952024-05-30T04:33:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16962024-05-30T04:33:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
16972024-05-30T04:33:02.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16982024-05-30T04:33:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16992024-05-30T04:33:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17002024-05-30T04:33:02.182ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17012024-05-30T04:33:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17022024-05-30T04:33:02.185ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
17032024-05-30T04:33:02.185ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
17042024-05-30T04:33:02.185ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
17052024-05-30T04:33:02.185ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 33.228253ms iteration = 6
17062024-05-30T04:33:02.188ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17072024-05-30T04:33:02.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17082024-05-30T04:33:02.189ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17092024-05-30T04:33:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17102024-05-30T04:33:02.196ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
17112024-05-30T04:33:02.196ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.785339ms iteration = 31
17122024-05-30T04:33:02.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17132024-05-30T04:33:02.197ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17142024-05-30T04:33:02.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17152024-05-30T04:33:02.199ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17162024-05-30T04:33:02.199ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17172024-05-30T04:33:02.199ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17182024-05-30T04:33:02.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17192024-05-30T04:33:02.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17202024-05-30T04:33:02.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17212024-05-30T04:33:02.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17222024-05-30T04:33:02.203ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17232024-05-30T04:33:02.203ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17242024-05-30T04:33:02.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17252024-05-30T04:33:02.205ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17262024-05-30T04:33:02.205ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17272024-05-30T04:33:02.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17282024-05-30T04:33:02.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17292024-05-30T04:33:02.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17302024-05-30T04:33:02.208ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17312024-05-30T04:33:02.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17322024-05-30T04:33:02.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17332024-05-30T04:33:02.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17342024-05-30T04:33:02.212ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17352024-05-30T04:33:02.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17362024-05-30T04:33:02.212ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
17372024-05-30T04:33:02.212ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 49.367028ms iteration = 6
17382024-05-30T04:33:02.212ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17392024-05-30T04:33:02.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17402024-05-30T04:33:02.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17412024-05-30T04:33:02.214ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17422024-05-30T04:33:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17432024-05-30T04:33:02.215ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17442024-05-30T04:33:02.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17452024-05-30T04:33:02.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
17462024-05-30T04:33:02.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17472024-05-30T04:33:02.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17482024-05-30T04:33:02.217ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17492024-05-30T04:33:02.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17502024-05-30T04:33:02.217ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 53.226492ms iteration = 11
17512024-05-30T04:33:02.218ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17522024-05-30T04:33:02.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17532024-05-30T04:33:02.220ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17542024-05-30T04:33:02.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17552024-05-30T04:33:02.220ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
17562024-05-30T04:33:02.220ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
17572024-05-30T04:33:02.220ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.025345ms iteration = 6
17582024-05-30T04:33:02.221ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17592024-05-30T04:33:02.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17602024-05-30T04:33:02.228ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17612024-05-30T04:33:02.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17622024-05-30T04:33:02.231ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17632024-05-30T04:33:02.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17642024-05-30T04:33:02.232ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17652024-05-30T04:33:02.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17662024-05-30T04:33:02.238ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17672024-05-30T04:33:02.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17682024-05-30T04:33:02.242ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17692024-05-30T04:33:02.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17702024-05-30T04:33:02.243ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17712024-05-30T04:33:02.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17722024-05-30T04:33:02.244ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17732024-05-30T04:33:02.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17742024-05-30T04:33:02.245ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17752024-05-30T04:33:02.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17762024-05-30T04:33:02.249ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17772024-05-30T04:33:02.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17782024-05-30T04:33:02.250ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17792024-05-30T04:33:02.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17802024-05-30T04:33:02.251ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17812024-05-30T04:33:02.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17822024-05-30T04:33:02.252ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17832024-05-30T04:33:02.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17842024-05-30T04:33:02.257ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17852024-05-30T04:33:02.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17862024-05-30T04:33:02.258ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
17872024-05-30T04:33:02.293ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
17882024-05-30T04:33:02.301ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
17892024-05-30T04:33:02.301ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17902024-05-30T04:33:02.301ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "59448c19-1bdc-439b-8718-21933ab8577b", "content-length": "26", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
17912024-05-30T04:33:02.301ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17922024-05-30T04:33:02.305ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17932024-05-30T04:33:02.305ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
17942024-05-30T04:33:02.305ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17952024-05-30T04:33:02.307ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "88b6f7ef-635d-4851-8f22-7e6b5e9c1610", "content-length": "26", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
17962024-05-30T04:33:02.307ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17972024-05-30T04:33:02.309ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
17982024-05-30T04:33:02.329ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
17992024-05-30T04:33:02.329ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18002024-05-30T04:33:02.329ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "273df25b-fff0-4040-9d9a-564a713c48b9", "content-length": "26", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18012024-05-30T04:33:02.329ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18022024-05-30T04:33:02.330ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
18032024-05-30T04:33:02.345ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
18042024-05-30T04:33:02.346ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b2a3ffc4-5180-4524-8d2f-fa673931b00a", "content-length": "691", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18052024-05-30T04:33:02.346ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
18062024-05-30T04:33:02.346ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
18072024-05-30T04:33:02.346ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
18082024-05-30T04:33:02.348ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 184.316751ms iteration = 11
18092024-05-30T04:33:02.400ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18102024-05-30T04:33:02.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18112024-05-30T04:33:02.402ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18122024-05-30T04:33:02.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18132024-05-30T04:33:02.403ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18142024-05-30T04:33:02.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18152024-05-30T04:33:02.405ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18162024-05-30T04:33:02.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18172024-05-30T04:33:02.407ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18182024-05-30T04:33:02.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18192024-05-30T04:33:02.408ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18202024-05-30T04:33:02.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18212024-05-30T04:33:02.408ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
18222024-05-30T04:33:02.409ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18232024-05-30T04:33:02.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18242024-05-30T04:33:02.413ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18252024-05-30T04:33:02.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18262024-05-30T04:33:02.418ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18272024-05-30T04:33:02.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18282024-05-30T04:33:02.419ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18292024-05-30T04:33:02.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18302024-05-30T04:33:02.421ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18312024-05-30T04:33:02.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18322024-05-30T04:33:02.422ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18332024-05-30T04:33:02.425ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18342024-05-30T04:33:02.425ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18352024-05-30T04:33:02.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18362024-05-30T04:33:02.439ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
18372024-05-30T04:33:02.439ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "545865b8-08a0-4f09-a8f9-f4089865c43c", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18382024-05-30T04:33:02.439ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18392024-05-30T04:33:02.440ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 1fbd52c8-c004-450c-ad40-e669ea7ac97a, kind: Service { id: b5bf7e90-33d3-4f67-b9d3-793143654e9a }, name: Name("nexus-b5bf7e90-33d3-4f67-b9d3-793143654e9a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 11])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9ad8a9b8-e418-45b0-906f-2b7edd8ecea2, kind: Service { id: 31c86263-9349-4603-aaa0-6dc44827df57 }, name: Name("nexus-31c86263-9349-4603-aaa0-6dc44827df57"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 132, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9d824d0b-7228-40a5-a231-be10889211c7, kind: Service { id: 7c996f98-0c86-4480-8700-580fc6f3590a }, name: Name("nexus-7c996f98-0c86-4480-8700-580fc6f3590a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 254, 127])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: e232750d-d52e-4ac4-8c87-d7b0e4145a1f, kind: Service { id: 8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3 }, name: Name("external-dns-8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 109])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: fb84270c-485b-4df1-a4a3-7bf184a5f27c, kind: Service { id: f72d0bad-b96e-4cb4-9157-126b67797b9b }, name: Name("external-dns-f72d0bad-b96e-4cb4-9157-126b67797b9b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
18402024-05-30T04:33:02.440ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18412024-05-30T04:33:02.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18422024-05-30T04:33:02.441ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18432024-05-30T04:33:02.442ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3d1bd534-e0b5-40d6-b7e9-b8e4c3b13335", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18442024-05-30T04:33:02.442ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18452024-05-30T04:33:02.442ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
18462024-05-30T04:33:02.442ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18472024-05-30T04:33:02.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18482024-05-30T04:33:02.445ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
18492024-05-30T04:33:02.446ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 282.895395ms iteration = 8
18502024-05-30T04:33:02.446ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 283.017326ms iteration = 11
18512024-05-30T04:33:02.448ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
18522024-05-30T04:33:02.448ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "184ff18e-8924-457b-ab00-4514cad6210c", "content-length": "1", "date": "Thu, 30 May 2024 04:33:01 GMT"} })
18532024-05-30T04:33:02.449ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
18542024-05-30T04:33:02.449ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: e232750d-d52e-4ac4-8c87-d7b0e4145a1f, kind: Service { id: 8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3 }, name: Name("external-dns-8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 109])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: fb84270c-485b-4df1-a4a3-7bf184a5f27c, kind: Service { id: f72d0bad-b96e-4cb4-9157-126b67797b9b }, name: Name("external-dns-f72d0bad-b96e-4cb4-9157-126b67797b9b"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 217, 61])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }] }, VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 1fbd52c8-c004-450c-ad40-e669ea7ac97a, kind: Service { id: b5bf7e90-33d3-4f67-b9d3-793143654e9a }, name: Name("nexus-b5bf7e90-33d3-4f67-b9d3-793143654e9a"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 249, 11])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9ad8a9b8-e418-45b0-906f-2b7edd8ecea2, kind: Service { id: 31c86263-9349-4603-aaa0-6dc44827df57 }, name: Name("nexus-31c86263-9349-4603-aaa0-6dc44827df57"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 132, 40])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 9d824d0b-7228-40a5-a231-be10889211c7, kind: Service { id: 7c996f98-0c86-4480-8700-580fc6f3590a }, name: Name("nexus-7c996f98-0c86-4480-8700-580fc6f3590a"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 254, 127])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0 }] }]
18552024-05-30T04:33:02.449ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
18562024-05-30T04:33:02.450ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
18572024-05-30T04:33:02.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
18582024-05-30T04:33:02.452ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 289.201328ms iteration = 11
18592024-05-30T04:33:02.474ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
18602024-05-30T04:33:02.475ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b9b25c52-8d90-4e47-85a3-e74511d5ed95", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18612024-05-30T04:33:02.475ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18622024-05-30T04:33:02.476ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "eb0b39e9-1032-4167-96b5-81e93acea538", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18632024-05-30T04:33:02.476ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18642024-05-30T04:33:02.478ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "db177e28-5afa-4411-9d44-9bc0268fd5ac", "date": "Thu, 30 May 2024 04:33:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18652024-05-30T04:33:02.478ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18662024-05-30T04:33:02.480ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
18672024-05-30T04:33:02.480ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 327.898593ms iteration = 8
18682024-05-30T04:33:02.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18692024-05-30T04:33:02.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18702024-05-30T04:33:02.538ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
18712024-05-30T04:33:02.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18722024-05-30T04:33:02.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18732024-05-30T04:33:02.545ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
18742024-05-30T04:33:02.546ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85b0866c-9101-4791-a0c0-199386166578", "content-length": "121", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18752024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
18762024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
18772024-05-30T04:33:02.546ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
18782024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
18792024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
18802024-05-30T04:33:02.546ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
18812024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
18822024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
18832024-05-30T04:33:02.546ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
18842024-05-30T04:33:02.546ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18852024-05-30T04:33:02.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18862024-05-30T04:33:02.548ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
18872024-05-30T04:33:02.548ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "af5af5af-1912-469d-a674-609f1b1848ac", "content-length": "2", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18882024-05-30T04:33:02.548ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
18892024-05-30T04:33:02.548ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5d579be5-0858-49ec-9209-8f9eef9c57b8", "content-length": "52", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18902024-05-30T04:33:02.548ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
18912024-05-30T04:33:02.548ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
18922024-05-30T04:33:02.601ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e2f6c7a2-e2c3-472c-8db8-b335d0ac3c63", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18932024-05-30T04:33:02.601ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
18942024-05-30T04:33:02.601ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
18952024-05-30T04:33:02.602ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1e901947-5409-47cf-96f6-9210eedc7b8c", "content-length": "467", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
18962024-05-30T04:33:02.602ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18972024-05-30T04:33:02.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
18982024-05-30T04:33:02.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
18992024-05-30T04:33:02.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19002024-05-30T04:33:02.609ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19012024-05-30T04:33:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19022024-05-30T04:33:02.611ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19032024-05-30T04:33:02.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19042024-05-30T04:33:02.618ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 453.750231ms iteration = 11
19052024-05-30T04:33:02.677ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 73ed95a1-f279-488a-9386-6d3e4371db4d, time_created: 2024-05-30T04:28:21.534950Z, time_modified: 2024-05-30T04:28:21.534950Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: 0, usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
19062024-05-30T04:33:02.677ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
19072024-05-30T04:33:02.679ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19082024-05-30T04:33:02.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19092024-05-30T04:33:02.683ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19102024-05-30T04:33:02.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19112024-05-30T04:33:02.685ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122024-05-30T04:33:02.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
19132024-05-30T04:33:02.718ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19142024-05-30T04:33:02.719ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e1797dda-9c0e-4575-9c89-6a5cd13cbe08", "date": "Thu, 30 May 2024 04:33:02 GMT"} })
19152024-05-30T04:33:02.721ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:487
19162024-05-30T04:33:02.721ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 556.807886ms
19172024-05-30T04:33:02.721ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 556.815406ms iteration = 2
19182024-05-30T04:33:02.772ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 149 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 38c12463-c6e2-4a4e-a91a-5ede783418ff response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
19192024-05-30T04:33:12.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
19202024-05-30T04:33:12.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19212024-05-30T04:33:12.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19222024-05-30T04:33:12.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19232024-05-30T04:33:12.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19242024-05-30T04:33:12.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19252024-05-30T04:33:12.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19262024-05-30T04:33:12.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19272024-05-30T04:33:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19282024-05-30T04:33:12.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19292024-05-30T04:33:12.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
19302024-05-30T04:33:12.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19312024-05-30T04:33:12.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19322024-05-30T04:33:12.178ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
19332024-05-30T04:33:12.178ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.024843ms iteration = 32
19342024-05-30T04:33:12.772ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 142 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 8577947a-b87d-4810-90bc-51b7e470ed40 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
19352024-05-30T04:33:22.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
19362024-05-30T04:33:22.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19372024-05-30T04:33:22.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19382024-05-30T04:33:22.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19392024-05-30T04:33:22.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19402024-05-30T04:33:22.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19412024-05-30T04:33:22.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19422024-05-30T04:33:22.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19432024-05-30T04:33:22.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19442024-05-30T04:33:22.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19452024-05-30T04:33:22.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
19462024-05-30T04:33:22.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19472024-05-30T04:33:22.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19482024-05-30T04:33:22.178ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
19492024-05-30T04:33:22.178ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.115084ms iteration = 33
19502024-05-30T04:33:22.773ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 148 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 27858a1f-36da-4302-9033-257bbd56248a response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
19512024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
19522024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
19532024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19542024-05-30T04:33:32.164ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
19552024-05-30T04:33:32.164ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
19562024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 15.191\u{b5}s iteration = 12
19572024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
19582024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
19592024-05-30T04:33:32.164ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
19602024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
19612024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
19622024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
19632024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19642024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19652024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
19662024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
19672024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19682024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19692024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
19702024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19712024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
19722024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19732024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19742024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19752024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
19762024-05-30T04:33:32.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19772024-05-30T04:33:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19782024-05-30T04:33:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19792024-05-30T04:33:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19802024-05-30T04:33:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
19812024-05-30T04:33:32.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822024-05-30T04:33:32.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19832024-05-30T04:33:32.167ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19842024-05-30T04:33:32.167ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19852024-05-30T04:33:32.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19862024-05-30T04:33:32.168ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
19872024-05-30T04:33:32.168ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.646982ms iteration = 12
19882024-05-30T04:33:32.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1234, tv_nsec: 280675275 } })
19892024-05-30T04:33:32.169ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1234, tv_nsec: 280822607 } })
19902024-05-30T04:33:32.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
19912024-05-30T04:33:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
19922024-05-30T04:33:32.170ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19932024-05-30T04:33:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19942024-05-30T04:33:32.170ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19952024-05-30T04:33:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
19962024-05-30T04:33:32.170ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
19972024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19982024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
19992024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20002024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20012024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20022024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20032024-05-30T04:33:32.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20042024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20052024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20062024-05-30T04:33:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20072024-05-30T04:33:32.173ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
20082024-05-30T04:33:32.173ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
20092024-05-30T04:33:32.173ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 8.484298ms iteration = 12
20102024-05-30T04:33:32.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20112024-05-30T04:33:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20122024-05-30T04:33:32.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20132024-05-30T04:33:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20142024-05-30T04:33:32.173ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1234, tv_nsec: 285667463 } })
20152024-05-30T04:33:32.174ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
20162024-05-30T04:33:32.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20172024-05-30T04:33:32.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
20182024-05-30T04:33:32.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20192024-05-30T04:33:32.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20202024-05-30T04:33:32.178ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
20212024-05-30T04:33:32.178ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.067802ms iteration = 12
20222024-05-30T04:33:32.182ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
20232024-05-30T04:33:32.182ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.321531ms iteration = 34
20242024-05-30T04:33:32.185ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20252024-05-30T04:33:32.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20262024-05-30T04:33:32.185ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20272024-05-30T04:33:32.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20282024-05-30T04:33:32.186ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20292024-05-30T04:33:32.187ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20302024-05-30T04:33:32.187ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20312024-05-30T04:33:32.187ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20322024-05-30T04:33:32.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20332024-05-30T04:33:32.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20342024-05-30T04:33:32.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20352024-05-30T04:33:32.189ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20362024-05-30T04:33:32.190ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
20372024-05-30T04:33:32.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20382024-05-30T04:33:32.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20392024-05-30T04:33:32.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
20402024-05-30T04:33:32.191ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20412024-05-30T04:33:32.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20422024-05-30T04:33:32.192ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.007052ms iteration = 12
20432024-05-30T04:33:32.265ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
20442024-05-30T04:33:32.265ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20452024-05-30T04:33:32.266ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
20462024-05-30T04:33:32.266ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20472024-05-30T04:33:32.266ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "071ad3cb-e05c-4ca1-aca3-24e32e4fb448", "content-length": "26", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
20482024-05-30T04:33:32.266ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20492024-05-30T04:33:32.267ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9bcaa4d9-51f3-4f19-8c77-6e917bd744a5", "content-length": "26", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
20502024-05-30T04:33:32.267ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20512024-05-30T04:33:32.269ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20522024-05-30T04:33:32.270ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20532024-05-30T04:33:32.280ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
20542024-05-30T04:33:32.280ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20552024-05-30T04:33:32.280ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7caa8994-2017-4ca6-87c5-45c6e0ec4136", "content-length": "26", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
20562024-05-30T04:33:32.280ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20572024-05-30T04:33:32.282ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
20582024-05-30T04:33:32.301ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
20592024-05-30T04:33:32.302ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2310bd36-9b85-4023-b121-c9c95780442a", "content-length": "691", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
20602024-05-30T04:33:32.302ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
20612024-05-30T04:33:32.302ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
20622024-05-30T04:33:32.302ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
20632024-05-30T04:33:32.303ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 139.330734ms iteration = 12
20642024-05-30T04:33:32.347ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
20652024-05-30T04:33:32.347ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "19c93ba7-66a8-476a-90ec-168c17a3400a", "content-length": "1", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
20662024-05-30T04:33:32.348ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20672024-05-30T04:33:32.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20682024-05-30T04:33:32.350ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 186.254743ms iteration = 12
20692024-05-30T04:33:32.358ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 193.8417ms iteration = 12
20702024-05-30T04:33:32.417ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20712024-05-30T04:33:32.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20722024-05-30T04:33:32.418ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
20732024-05-30T04:33:32.418ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20742024-05-30T04:33:32.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20752024-05-30T04:33:32.427ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
20762024-05-30T04:33:32.427ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8e13bb23-514f-4bd9-9e3b-e521388a49fa", "content-length": "121", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
20772024-05-30T04:33:32.427ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
20782024-05-30T04:33:32.427ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
20792024-05-30T04:33:32.428ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
20802024-05-30T04:33:32.428ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
20812024-05-30T04:33:32.428ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
20822024-05-30T04:33:32.428ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
20832024-05-30T04:33:32.428ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
20842024-05-30T04:33:32.428ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
20852024-05-30T04:33:32.428ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
20862024-05-30T04:33:32.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20872024-05-30T04:33:32.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20882024-05-30T04:33:32.429ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
20892024-05-30T04:33:32.430ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1438e073-a4db-425c-b9da-6ff68dc74189", "content-length": "2", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
20902024-05-30T04:33:32.430ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
20912024-05-30T04:33:32.430ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e22865b1-b6ab-482f-983e-59e865201031", "content-length": "52", "date": "Thu, 30 May 2024 04:33:31 GMT"} })
20922024-05-30T04:33:32.430ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
20932024-05-30T04:33:32.430ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20942024-05-30T04:33:32.510ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d2ad98c5-2a8b-403f-bacc-8265d6458fd6", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
20952024-05-30T04:33:32.510ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
20962024-05-30T04:33:32.510ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20972024-05-30T04:33:32.511ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "089a132b-9595-414b-938d-3d11a5bed307", "content-length": "467", "date": "Thu, 30 May 2024 04:33:32 GMT"} })
20982024-05-30T04:33:32.511ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20992024-05-30T04:33:32.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21002024-05-30T04:33:32.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21012024-05-30T04:33:32.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21022024-05-30T04:33:32.517ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21032024-05-30T04:33:32.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21042024-05-30T04:33:32.518ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21052024-05-30T04:33:32.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21062024-05-30T04:33:32.525ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 361.295917ms iteration = 12
21072024-05-30T04:33:32.773ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 821 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = fdbf9537-3d0d-4076-acee-c36461fa4827 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
21082024-05-30T04:33:42.163ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
21092024-05-30T04:33:42.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21102024-05-30T04:33:42.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21112024-05-30T04:33:42.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21122024-05-30T04:33:42.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21132024-05-30T04:33:42.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21142024-05-30T04:33:42.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21152024-05-30T04:33:42.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21162024-05-30T04:33:42.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21172024-05-30T04:33:42.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21182024-05-30T04:33:42.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
21192024-05-30T04:33:42.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21202024-05-30T04:33:42.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21212024-05-30T04:33:42.178ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
21222024-05-30T04:33:42.178ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.493666ms iteration = 35
21232024-05-30T04:33:42.774ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 821 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 83742276-26c2-42dc-bf1e-17edaebdd036 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
21242024-05-30T04:33:52.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
21252024-05-30T04:33:52.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21262024-05-30T04:33:52.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21272024-05-30T04:33:52.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21282024-05-30T04:33:52.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21292024-05-30T04:33:52.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21302024-05-30T04:33:52.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21312024-05-30T04:33:52.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21322024-05-30T04:33:52.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21332024-05-30T04:33:52.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21342024-05-30T04:33:52.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
21352024-05-30T04:33:52.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21362024-05-30T04:33:52.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21372024-05-30T04:33:52.178ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
21382024-05-30T04:33:52.178ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.331945ms iteration = 36
21392024-05-30T04:33:52.773ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 141 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 07c954bb-992d-4c6e-a6cc-d0d2b86fc580 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
21402024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
21412024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
21422024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
21432024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
21442024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21452024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
21462024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
21472024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
21482024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21492024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21502024-05-30T04:34:02.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21512024-05-30T04:34:02.153ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 265299469 } })
21522024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21532024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21542024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
21552024-05-30T04:34:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
21562024-05-30T04:34:02.155ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 266586404 } })
21572024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21582024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21592024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
21602024-05-30T04:34:02.156ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.030055ms iteration = 7
21612024-05-30T04:34:02.159ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
21622024-05-30T04:34:02.159ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.820317ms iteration = 7
21632024-05-30T04:34:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
21642024-05-30T04:34:02.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21652024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
21662024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
21672024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
21682024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21692024-05-30T04:34:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21702024-05-30T04:34:02.162ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21712024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
21722024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21732024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
21742024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
21752024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
21762024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
21772024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
21782024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
21792024-05-30T04:34:02.164ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
21802024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.44\u{b5}s iteration = 8
21812024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21822024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21832024-05-30T04:34:02.164ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
21842024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
21852024-05-30T04:34:02.164ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:24:02.164556797 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
21862024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
21872024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21882024-05-30T04:34:02.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21892024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21902024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
21912024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
21922024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21932024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
21942024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
21952024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21962024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21972024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
21982024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
21992024-05-30T04:34:02.165ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
22002024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
22012024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22022024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
22032024-05-30T04:34:02.165ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
22042024-05-30T04:34:02.165ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
22052024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.46\u{b5}s iteration = 13
22062024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
22072024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
22082024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22092024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22102024-05-30T04:34:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22112024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22122024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22132024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22142024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
22152024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22162024-05-30T04:34:02.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22172024-05-30T04:34:02.166ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 278192198 } })
22182024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22192024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22202024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22212024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22222024-05-30T04:34:02.167ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
22232024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.114926ms iteration = 13
22242024-05-30T04:34:02.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22252024-05-30T04:34:02.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 279501913 } })
22262024-05-30T04:34:02.168ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
22272024-05-30T04:34:02.169ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22282024-05-30T04:34:02.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
22292024-05-30T04:34:02.169ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22302024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22312024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
22322024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
22332024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22342024-05-30T04:34:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22352024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22362024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22372024-05-30T04:34:02.171ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:24:02.164556797 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
22382024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.466414ms iteration = 7
22392024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22402024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22412024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22422024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22432024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22442024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22452024-05-30T04:34:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22462024-05-30T04:34:02.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22472024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
22482024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22492024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22502024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22512024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22522024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22532024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22542024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22552024-05-30T04:34:02.173ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1264, tv_nsec: 285079007 } })
22562024-05-30T04:34:02.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22572024-05-30T04:34:02.174ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
22582024-05-30T04:34:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22592024-05-30T04:34:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22602024-05-30T04:34:02.174ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22612024-05-30T04:34:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22622024-05-30T04:34:02.174ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
22632024-05-30T04:34:02.174ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
22642024-05-30T04:34:02.174ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 9.289767ms iteration = 7
22652024-05-30T04:34:02.175ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22662024-05-30T04:34:02.175ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
22672024-05-30T04:34:02.175ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
22682024-05-30T04:34:02.175ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 9.968595ms iteration = 13
22692024-05-30T04:34:02.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22702024-05-30T04:34:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22712024-05-30T04:34:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22722024-05-30T04:34:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
22732024-05-30T04:34:02.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
22742024-05-30T04:34:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
22752024-05-30T04:34:02.178ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
22762024-05-30T04:34:02.178ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
22772024-05-30T04:34:02.178ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
22782024-05-30T04:34:02.178ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.597823ms iteration = 7
22792024-05-30T04:34:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
22802024-05-30T04:34:02.179ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22812024-05-30T04:34:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22822024-05-30T04:34:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
22832024-05-30T04:34:02.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22842024-05-30T04:34:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22852024-05-30T04:34:02.180ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
22862024-05-30T04:34:02.180ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 15.129254ms iteration = 13
22872024-05-30T04:34:02.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22882024-05-30T04:34:02.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22892024-05-30T04:34:02.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22902024-05-30T04:34:02.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
22912024-05-30T04:34:02.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
22922024-05-30T04:34:02.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
22932024-05-30T04:34:02.186ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
22942024-05-30T04:34:02.186ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.041254ms iteration = 37
22952024-05-30T04:34:02.188ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22962024-05-30T04:34:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22972024-05-30T04:34:02.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22982024-05-30T04:34:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22992024-05-30T04:34:02.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23002024-05-30T04:34:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23012024-05-30T04:34:02.191ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23022024-05-30T04:34:02.191ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23032024-05-30T04:34:02.192ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) }
23042024-05-30T04:34:02.192ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 28.223465ms iteration = 7
23052024-05-30T04:34:02.193ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23062024-05-30T04:34:02.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23072024-05-30T04:34:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23082024-05-30T04:34:02.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
23092024-05-30T04:34:02.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23102024-05-30T04:34:02.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23112024-05-30T04:34:02.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23122024-05-30T04:34:02.199ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.440035ms iteration = 13
23132024-05-30T04:34:02.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23142024-05-30T04:34:02.200ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23152024-05-30T04:34:02.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23162024-05-30T04:34:02.203ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
23172024-05-30T04:34:02.203ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
23182024-05-30T04:34:02.203ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
23192024-05-30T04:34:02.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23202024-05-30T04:34:02.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23212024-05-30T04:34:02.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23222024-05-30T04:34:02.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23232024-05-30T04:34:02.208ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23242024-05-30T04:34:02.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23252024-05-30T04:34:02.211ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
23262024-05-30T04:34:02.211ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
23272024-05-30T04:34:02.211ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23282024-05-30T04:34:02.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23292024-05-30T04:34:02.212ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23302024-05-30T04:34:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23312024-05-30T04:34:02.214ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23322024-05-30T04:34:02.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23332024-05-30T04:34:02.216ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
23342024-05-30T04:34:02.216ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
23352024-05-30T04:34:02.216ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
23362024-05-30T04:34:02.216ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 63.282678ms iteration = 7
23372024-05-30T04:34:02.264ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
23382024-05-30T04:34:02.324ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
23392024-05-30T04:34:02.329ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
23402024-05-30T04:34:02.329ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23412024-05-30T04:34:02.331ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9cf50a0c-4105-4bf4-9944-a6c3db3b4aac", "content-length": "26", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
23422024-05-30T04:34:02.331ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23432024-05-30T04:34:02.334ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23442024-05-30T04:34:02.343ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
23452024-05-30T04:34:02.343ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23462024-05-30T04:34:02.343ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ac33ca-7d18-47da-aaee-57d7b6d534de", "content-length": "26", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
23472024-05-30T04:34:02.343ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23482024-05-30T04:34:02.345ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23492024-05-30T04:34:02.437ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
23502024-05-30T04:34:02.437ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23512024-05-30T04:34:02.437ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ba825e7f-5c40-4af7-9aee-eafe887fbec9", "content-length": "26", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
23522024-05-30T04:34:02.437ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23532024-05-30T04:34:02.440ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
23542024-05-30T04:34:02.445ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
23552024-05-30T04:34:02.491ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
23562024-05-30T04:34:02.491ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b87de5bf-bcc5-416b-bb0c-821706a60ed5", "content-length": "691", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
23572024-05-30T04:34:02.491ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
23582024-05-30T04:34:02.491ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
23592024-05-30T04:34:02.491ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
23602024-05-30T04:34:02.492ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 326.589217ms iteration = 13
23612024-05-30T04:34:02.565ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
23622024-05-30T04:34:02.565ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "28016c5d-915a-4571-b71a-7d7be7f2b094", "content-length": "1", "date": "Thu, 30 May 2024 04:34:01 GMT"} })
23632024-05-30T04:34:02.566ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23642024-05-30T04:34:02.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23652024-05-30T04:34:02.568ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 404.13793ms iteration = 13
23662024-05-30T04:34:02.590ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
23672024-05-30T04:34:02.591ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "47ef2c24-0fc5-4575-b69c-8a3cb784d3c1", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23682024-05-30T04:34:02.591ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23692024-05-30T04:34:02.593ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e3ff0789-c43e-4fe8-8f60-65e7b0bd03e9", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23702024-05-30T04:34:02.593ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23712024-05-30T04:34:02.594ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f67b6766-fcf0-4ef7-97f5-eeb10e311076", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23722024-05-30T04:34:02.594ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23732024-05-30T04:34:02.596ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
23742024-05-30T04:34:02.596ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 442.907496ms iteration = 9
23752024-05-30T04:34:02.618ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
23762024-05-30T04:34:02.618ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1aa60bb7-9688-4689-9205-e29c030a1984", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23772024-05-30T04:34:02.618ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23782024-05-30T04:34:02.621ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "26365548-59a8-48d9-97e4-543fe4a1cc64", "date": "Thu, 30 May 2024 04:34:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23792024-05-30T04:34:02.621ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23802024-05-30T04:34:02.623ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 458.814889ms iteration = 13
23812024-05-30T04:34:02.624ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
23822024-05-30T04:34:02.624ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 459.965872ms iteration = 9
23832024-05-30T04:34:02.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23842024-05-30T04:34:02.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23852024-05-30T04:34:02.684ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
23862024-05-30T04:34:02.684ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23872024-05-30T04:34:02.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23882024-05-30T04:34:02.691ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23892024-05-30T04:34:02.692ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ca0eb78-8410-4818-aa2e-3d1a49aa87cd", "content-length": "121", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
23902024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
23912024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
23922024-05-30T04:34:02.692ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
23932024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
23942024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
23952024-05-30T04:34:02.692ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
23962024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
23972024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
23982024-05-30T04:34:02.692ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
23992024-05-30T04:34:02.692ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24002024-05-30T04:34:02.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24012024-05-30T04:34:02.694ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
24022024-05-30T04:34:02.695ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ff4f297c-a56f-421f-b75a-f313dae69111", "content-length": "2", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24032024-05-30T04:34:02.695ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
24042024-05-30T04:34:02.695ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8763a459-a252-4ff6-8da2-47a4e8e172e7", "content-length": "52", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24052024-05-30T04:34:02.695ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
24062024-05-30T04:34:02.695ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24072024-05-30T04:34:02.743ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6d93ad8e-b86b-4d27-ae48-d74b61742cdb", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24082024-05-30T04:34:02.743ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
24092024-05-30T04:34:02.743ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24102024-05-30T04:34:02.752ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cbf25117-5a7d-4455-b20c-faba7b0bcb02", "content-length": "467", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24112024-05-30T04:34:02.752ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24122024-05-30T04:34:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24132024-05-30T04:34:02.755ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24142024-05-30T04:34:02.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24152024-05-30T04:34:02.758ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24162024-05-30T04:34:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24172024-05-30T04:34:02.759ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24182024-05-30T04:34:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24192024-05-30T04:34:02.766ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 600.919334ms iteration = 13
24202024-05-30T04:34:02.773ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 137 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = a3913363-c0f4-495d-b0a9-d3ccee577e00 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
24212024-05-30T04:34:02.776ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
24222024-05-30T04:34:02.776ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
24232024-05-30T04:34:02.806ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
24242024-05-30T04:34:02.806ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:59748
24252024-05-30T04:34:02.806ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:59748 req_id = c9cf24a9-90a5-4462-96af-fd326f4327b2 roles = RoleSet { roles: {} } uri = /metrics/producers
24262024-05-30T04:34:02.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:59748 req_id = c9cf24a9-90a5-4462-96af-fd326f4327b2 resource = Database result = Ok(()) uri = /metrics/producers
24272024-05-30T04:34:02.838ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
24282024-05-30T04:34:02.838ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:59748 req_id = c9cf24a9-90a5-4462-96af-fd326f4327b2 roles = RoleSet { roles: {} } uri = /metrics/producers
24292024-05-30T04:34:02.838ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:59748 req_id = c9cf24a9-90a5-4462-96af-fd326f4327b2 resource = Database result = Ok(()) uri = /metrics/producers
24302024-05-30T04:34:02.867ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
24312024-05-30T04:34:02.868ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3fff39e2-d795-47ed-8ac9-a1759ac9558a", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24322024-05-30T04:34:02.868ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = b5bf7e90-33d3-4f67-b9d3-793143654e9a
24332024-05-30T04:34:02.869ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 62759 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:59748 req_id = c9cf24a9-90a5-4462-96af-fd326f4327b2 response_code = 201 uri = /metrics/producers
24342024-05-30T04:34:02.869ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::b)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "c9cf24a9-90a5-4462-96af-fd326f4327b2", "content-length": "41", "date": "Thu, 30 May 2024 04:34:02 GMT"} })
24352024-05-30T04:34:02.870ZDEBGnexus (producer-registration-task): registered with nexus successfully
24362024-05-30T04:34:02.870ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
24372024-05-30T04:34:12.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
24382024-05-30T04:34:12.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24392024-05-30T04:34:12.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24402024-05-30T04:34:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24412024-05-30T04:34:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24422024-05-30T04:34:12.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24432024-05-30T04:34:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24442024-05-30T04:34:12.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24452024-05-30T04:34:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24462024-05-30T04:34:12.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24472024-05-30T04:34:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
24482024-05-30T04:34:12.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24492024-05-30T04:34:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24502024-05-30T04:34:12.180ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
24512024-05-30T04:34:12.180ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.150604ms iteration = 38
24522024-05-30T04:34:12.869ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 246 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = a3eb9ba7-812b-4598-9c7a-cc1b669f2cd6 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
24532024-05-30T04:34:22.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
24542024-05-30T04:34:22.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24552024-05-30T04:34:22.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24562024-05-30T04:34:22.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24572024-05-30T04:34:22.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24582024-05-30T04:34:22.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24592024-05-30T04:34:22.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24602024-05-30T04:34:22.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24612024-05-30T04:34:22.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24622024-05-30T04:34:22.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24632024-05-30T04:34:22.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
24642024-05-30T04:34:22.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24652024-05-30T04:34:22.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24662024-05-30T04:34:22.179ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
24672024-05-30T04:34:22.179ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.157714ms iteration = 39
24682024-05-30T04:34:22.870ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 239 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 04690d42-981d-46a0-a06c-cd1fb9a7a6bb response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
24692024-05-30T04:34:29.203ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:766 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1:0]:45902
24702024-05-30T04:34:29.203ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:45902 req_id = e2f5f381-a788-4e55-98b8-0255398378fb roles = RoleSet { roles: {} } uri = /metrics/producers
24712024-05-30T04:34:29.204ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:45902 req_id = e2f5f381-a788-4e55-98b8-0255398378fb resource = Database result = Ok(()) uri = /metrics/producers
24722024-05-30T04:34:29.237ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = 99d500db-1a9c-4b1e-9051-e0be361f71a9
24732024-05-30T04:34:29.237ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:45902 req_id = e2f5f381-a788-4e55-98b8-0255398378fb roles = RoleSet { roles: {} } uri = /metrics/producers
24742024-05-30T04:34:29.237ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:45902 req_id = e2f5f381-a788-4e55-98b8-0255398378fb resource = Database result = Ok(()) uri = /metrics/producers
24752024-05-30T04:34:29.242ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 uri = http://[fd00:1122:3344:101::d]:12223/producers
24762024-05-30T04:34:29.242ZDEBGnexus: client response oximeter-collector = 99d500db-1a9c-4b1e-9051-e0be361f71a9 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "616ed3ea-d024-4789-be47-801dcea1e7d1", "date": "Thu, 30 May 2024 04:34:29 GMT"} })
24772024-05-30T04:34:29.242ZINFOnexus: assigned collector to new producer collector_id = 99d500db-1a9c-4b1e-9051-e0be361f71a9 file = nexus/src/app/oximeter.rs:125 producer_id = c3e31699-e123-4b80-8462-b2fb4fd4546f
24782024-05-30T04:34:29.245ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 41619 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:45902 req_id = e2f5f381-a788-4e55-98b8-0255398378fb response_code = 201 uri = /metrics/producers
24792024-05-30T04:34:32.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
24802024-05-30T04:34:32.164ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
24812024-05-30T04:34:32.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24822024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24832024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
24842024-05-30T04:34:32.165ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
24852024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
24862024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24872024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
24882024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24892024-05-30T04:34:32.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24902024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
24912024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
24922024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
24932024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24942024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24952024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
24962024-05-30T04:34:32.166ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
24972024-05-30T04:34:32.166ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
24982024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.98\u{b5}s iteration = 14
24992024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25002024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
25012024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
25022024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25032024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25042024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25052024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25062024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25072024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25082024-05-30T04:34:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25092024-05-30T04:34:32.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25102024-05-30T04:34:32.167ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
25112024-05-30T04:34:32.167ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.65298ms iteration = 14
25122024-05-30T04:34:32.168ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25132024-05-30T04:34:32.168ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25142024-05-30T04:34:32.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1294, tv_nsec: 279528993 } })
25152024-05-30T04:34:32.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1294, tv_nsec: 279682055 } })
25162024-05-30T04:34:32.169ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25172024-05-30T04:34:32.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25182024-05-30T04:34:32.170ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
25192024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25202024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25212024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25222024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25232024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25242024-05-30T04:34:32.172ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
25252024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
25262024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25272024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25282024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25292024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25302024-05-30T04:34:32.172ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25312024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25322024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
25332024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25342024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
25352024-05-30T04:34:32.173ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25362024-05-30T04:34:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25372024-05-30T04:34:32.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25382024-05-30T04:34:32.174ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
25392024-05-30T04:34:32.174ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
25402024-05-30T04:34:32.174ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 7.82595ms iteration = 14
25412024-05-30T04:34:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25422024-05-30T04:34:32.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25432024-05-30T04:34:32.178ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1294, tv_nsec: 289285055 } })
25442024-05-30T04:34:32.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
25452024-05-30T04:34:32.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
25462024-05-30T04:34:32.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25472024-05-30T04:34:32.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25482024-05-30T04:34:32.196ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
25492024-05-30T04:34:32.196ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.364351ms iteration = 40
25502024-05-30T04:34:32.196ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25512024-05-30T04:34:32.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25522024-05-30T04:34:32.198ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25532024-05-30T04:34:32.199ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25542024-05-30T04:34:32.199ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25552024-05-30T04:34:32.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25562024-05-30T04:34:32.200ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25572024-05-30T04:34:32.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25582024-05-30T04:34:32.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25592024-05-30T04:34:32.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25602024-05-30T04:34:32.203ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25612024-05-30T04:34:32.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25622024-05-30T04:34:32.204ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25632024-05-30T04:34:32.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25642024-05-30T04:34:32.206ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25652024-05-30T04:34:32.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
25662024-05-30T04:34:32.208ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25672024-05-30T04:34:32.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25682024-05-30T04:34:32.209ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 43.539701ms iteration = 14
25692024-05-30T04:34:32.292ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
25702024-05-30T04:34:32.292ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25712024-05-30T04:34:32.293ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "39cc1c8a-585a-491c-9c0f-eed1c75c486d", "content-length": "26", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
25722024-05-30T04:34:32.293ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25732024-05-30T04:34:32.294ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25742024-05-30T04:34:32.337ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
25752024-05-30T04:34:32.337ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25762024-05-30T04:34:32.339ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3f9120cd-6827-4d92-9b0e-84f4dfc99f35", "content-length": "26", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
25772024-05-30T04:34:32.339ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25782024-05-30T04:34:32.340ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
25792024-05-30T04:34:32.354ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
25802024-05-30T04:34:32.354ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "db18d23c-a64a-4d8e-8c55-3f7a7e344d59", "content-length": "811", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
25812024-05-30T04:34:32.354ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 241, 76, 112])), vni: Vni(14850883) }} sled = buskin
25822024-05-30T04:34:32.354ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
25832024-05-30T04:34:32.354ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
25842024-05-30T04:34:32.357ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 190.814775ms iteration = 14
25852024-05-30T04:34:32.380ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
25862024-05-30T04:34:32.380ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25872024-05-30T04:34:32.380ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f5ddb7d7-13bf-48ba-b47f-acaff6597790", "content-length": "26", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
25882024-05-30T04:34:32.380ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25892024-05-30T04:34:32.382ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25902024-05-30T04:34:32.422ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/instances/c3e31699-e123-4b80-8462-b2fb4fd4546f/state
25912024-05-30T04:34:32.422ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = instance_watcher result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/instances/c3e31699-e123-4b80-8462-b2fb4fd4546f/state", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2482e6be-49d1-4f49-8a01-400e624a5624", "content-length": "313", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
25922024-05-30T04:34:32.428ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f state = Running
25932024-05-30T04:34:32.428ZINFOnexus (BackgroundTasks): received new runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:1982 instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f instance_state = InstanceRuntimeState { propolis_id: Some(5976ce1d-b479-45e7-940d-c7bd3ba7a7cb), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2024-05-30T04:34:21.668246734Z } propolis_id = 5976ce1d-b479-45e7-940d-c7bd3ba7a7cb vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2024-05-30T04:34:29.283435115Z }
25942024-05-30T04:34:32.428ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
25952024-05-30T04:34:32.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
25962024-05-30T04:34:32.432ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
25972024-05-30T04:34:32.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
25982024-05-30T04:34:32.434ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
25992024-05-30T04:34:32.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
26002024-05-30T04:34:32.436ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
26012024-05-30T04:34:32.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
26022024-05-30T04:34:32.437ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
26032024-05-30T04:34:32.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
26042024-05-30T04:34:32.439ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {} }
26052024-05-30T04:34:32.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Database result = Ok(())
26062024-05-30T04:34:32.440ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26072024-05-30T04:34:32.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, lookup_type: ById(d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab) }, key: 44b9d9f4-dd81-49cb-9277-bb9d9e2cb6e9, lookup_type: ById(44b9d9f4-dd81-49cb-9277-bb9d9e2cb6e9) }, key: c3e31699-e123-4b80-8462-b2fb4fd4546f, lookup_type: ById(c3e31699-e123-4b80-8462-b2fb4fd4546f) } result = Ok(())
26082024-05-30T04:34:32.443ZDEBGnexus (BackgroundTasks): instance state generation already advanced, won't touch network config background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f
26092024-05-30T04:34:32.454ZINFOnexus (BackgroundTasks): instance and vmm updated by sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2116 instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f instance_updated = false propolis_id = 5976ce1d-b479-45e7-940d-c7bd3ba7a7cb vmm_updated = false
26102024-05-30T04:34:32.454ZDEBGnexus (BackgroundTasks): update successful background_task = instance_watcher instance_id = c3e31699-e123-4b80-8462-b2fb4fd4546f instance_updated = false state = Running vmm_updated = false
26112024-05-30T04:34:32.455ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1
26122024-05-30T04:34:32.455ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 289.529262ms iteration = 14
26132024-05-30T04:34:32.480ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
26142024-05-30T04:34:32.480ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0787e450-8bf1-4015-a44c-3775dfb4fe6d", "content-length": "2", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
26152024-05-30T04:34:32.481ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26162024-05-30T04:34:32.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26172024-05-30T04:34:32.484ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 319.176213ms iteration = 14
26182024-05-30T04:34:32.517ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 352.93551ms iteration = 14
26192024-05-30T04:34:32.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26202024-05-30T04:34:32.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26212024-05-30T04:34:32.555ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
26222024-05-30T04:34:32.555ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26232024-05-30T04:34:32.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26242024-05-30T04:34:32.564ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
26252024-05-30T04:34:32.564ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "19cc3311-381d-467a-b67b-0141107abe26", "content-length": "121", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
26262024-05-30T04:34:32.564ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
26272024-05-30T04:34:32.564ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
26282024-05-30T04:34:32.565ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
26292024-05-30T04:34:32.565ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
26302024-05-30T04:34:32.565ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
26312024-05-30T04:34:32.565ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
26322024-05-30T04:34:32.565ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
26332024-05-30T04:34:32.565ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
26342024-05-30T04:34:32.565ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
26352024-05-30T04:34:32.565ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26362024-05-30T04:34:32.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26372024-05-30T04:34:32.568ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
26382024-05-30T04:34:32.568ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6b946df6-33e6-4b6c-ad9b-cb41980d00a0", "content-length": "2", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
26392024-05-30T04:34:32.568ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
26402024-05-30T04:34:32.568ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "37683417-f31a-4579-98cd-77896db55fe5", "content-length": "52", "date": "Thu, 30 May 2024 04:34:31 GMT"} })
26412024-05-30T04:34:32.568ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
26422024-05-30T04:34:32.568ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26432024-05-30T04:34:32.620ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a80a9c0d-03f9-4868-9af5-e6d3bfd92961", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
26442024-05-30T04:34:32.620ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
26452024-05-30T04:34:32.620ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26462024-05-30T04:34:32.621ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c65471fc-67ed-41ef-8140-2539185b2e87", "content-length": "467", "date": "Thu, 30 May 2024 04:34:32 GMT"} })
26472024-05-30T04:34:32.621ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26482024-05-30T04:34:32.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26492024-05-30T04:34:32.624ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26502024-05-30T04:34:32.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26512024-05-30T04:34:32.627ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26522024-05-30T04:34:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26532024-05-30T04:34:32.628ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26542024-05-30T04:34:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26552024-05-30T04:34:32.633ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 467.499579ms iteration = 14
26562024-05-30T04:34:32.872ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 2805 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = b530ed82-20ab-4959-9029-8bb93bbbffa0 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
26572024-05-30T04:34:42.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
26582024-05-30T04:34:42.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26592024-05-30T04:34:42.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26602024-05-30T04:34:42.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26612024-05-30T04:34:42.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26622024-05-30T04:34:42.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26632024-05-30T04:34:42.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26642024-05-30T04:34:42.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26652024-05-30T04:34:42.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26662024-05-30T04:34:42.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26672024-05-30T04:34:42.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
26682024-05-30T04:34:42.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26692024-05-30T04:34:42.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26702024-05-30T04:34:42.179ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
26712024-05-30T04:34:42.179ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.73393ms iteration = 41
26722024-05-30T04:34:42.870ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 618 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 1c59bf9d-1c7e-4a5f-8960-bc0aebb0b7bd response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
26732024-05-30T04:34:52.164ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
26742024-05-30T04:34:52.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26752024-05-30T04:34:52.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26762024-05-30T04:34:52.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26772024-05-30T04:34:52.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26782024-05-30T04:34:52.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26792024-05-30T04:34:52.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26802024-05-30T04:34:52.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26812024-05-30T04:34:52.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26822024-05-30T04:34:52.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26832024-05-30T04:34:52.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
26842024-05-30T04:34:52.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26852024-05-30T04:34:52.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26862024-05-30T04:34:52.179ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
26872024-05-30T04:34:52.179ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.920891ms iteration = 42
26882024-05-30T04:34:52.871ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 1097 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 82657ce5-1a12-4c01-8229-fae4bceeac3e response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
26892024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
26902024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
26912024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
26922024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
26932024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
26942024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
26952024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26962024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26972024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
26982024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26992024-05-30T04:35:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27002024-05-30T04:35:02.155ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("36471b17-5c70-4d58-b316-111ada5cfd09.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("366f4458-e643-4fe9-8142-9de9376ebefb.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("685df176-1376-4b0f-9bdf-c7e3690ff5fa.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 265358870 } })
27012024-05-30T04:35:02.155ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8622daa2-0ad5-44d6-b2a1-9ec1ef3e9ea3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("f72d0bad-b96e-4cb4-9157-126b67797b9b.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 265425381 } })
27022024-05-30T04:35:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27032024-05-30T04:35:02.155ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27042024-05-30T04:35:02.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27052024-05-30T04:35:02.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27062024-05-30T04:35:02.156ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
27072024-05-30T04:35:02.156ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.045114ms iteration = 8
27082024-05-30T04:35:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27092024-05-30T04:35:02.157ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27102024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
27112024-05-30T04:35:02.159ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.862976ms iteration = 8
27122024-05-30T04:35:02.161ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27132024-05-30T04:35:02.162ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27142024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
27152024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
27162024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
27172024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27182024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27192024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
27202024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
27212024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
27222024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27232024-05-30T04:35:02.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27242024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27252024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27262024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
27272024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
27282024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27292024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
27302024-05-30T04:35:02.165ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-05-30 04:25:02.165423436 UTC file = nexus/src/app/background/metrics_producer_gc.rs:48
27312024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
27322024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
27332024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
27342024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27352024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27362024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
27372024-05-30T04:35:02.165ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
27382024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
27392024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
27402024-05-30T04:35:02.165ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = b9a32bb6-c543-452c-bbbf-8209a8288553
27412024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
27422024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.16\u{b5}s iteration = 9
27432024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
27442024-05-30T04:35:02.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27452024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27462024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
27472024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27482024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
27492024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27502024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
27512024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27522024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
27532024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27542024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
27552024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27562024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
27572024-05-30T04:35:02.166ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:75
27582024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
27592024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
27602024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27612024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
27622024-05-30T04:35:02.166ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
27632024-05-30T04:35:02.166ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
27642024-05-30T04:35:02.166ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.801\u{b5}s iteration = 15
27652024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27662024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27672024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27682024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
27692024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27702024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27712024-05-30T04:35:02.167ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 278200828 } })
27722024-05-30T04:35:02.167ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27732024-05-30T04:35:02.168ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
27742024-05-30T04:35:02.168ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-05-30 04:25:02.165423436 UTC file = nexus/src/app/background/metrics_producer_gc.rs:69 pruned = {}
27752024-05-30T04:35:02.168ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.257237ms iteration = 8
27762024-05-30T04:35:02.168ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27772024-05-30T04:35:02.169ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
27782024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.532521ms iteration = 15
27792024-05-30T04:35:02.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27802024-05-30T04:35:02.170ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 280468804 } })
27812024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27822024-05-30T04:35:02.170ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
27832024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27842024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27852024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27862024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
27872024-05-30T04:35:02.170ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/abandoned_vmm_reaper.rs:183 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
27882024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 3.987125ms iteration = 8
27892024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27902024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
27912024-05-30T04:35:02.170ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27922024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27932024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27942024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27952024-05-30T04:35:02.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27962024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
27972024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 0 version = 2
27982024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27992024-05-30T04:35:02.172ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28002024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28012024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28022024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28032024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28042024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28052024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28062024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
28072024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
28082024-05-30T04:35:02.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28092024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28102024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28112024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28122024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28132024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
28142024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28152024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28162024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28172024-05-30T04:35:02.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28182024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28192024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28202024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28212024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28222024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28232024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28242024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28252024-05-30T04:35:02.175ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28262024-05-30T04:35:02.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28272024-05-30T04:35:02.176ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
28282024-05-30T04:35:02.176ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
28292024-05-30T04:35:02.176ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 9.527359ms iteration = 15
28302024-05-30T04:35:02.176ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1324, tv_nsec: 286517124 } })
28312024-05-30T04:35:02.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28322024-05-30T04:35:02.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28332024-05-30T04:35:02.177ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
28342024-05-30T04:35:02.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28352024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
28362024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28372024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28382024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28392024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28402024-05-30T04:35:02.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28412024-05-30T04:35:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28422024-05-30T04:35:02.180ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28432024-05-30T04:35:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28442024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28452024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28462024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = fd02df78-14e6-48df-8af2-a3a71f4538a3 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
28472024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
28482024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-05-30 04:28:26.452464 UTC dns_group = internal generation = 1
28492024-05-30T04:35:02.181ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 27.302544ms iteration = 8
28502024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
28512024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28522024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a0eaabf0-32d0-4aa9-87f7-e9444bf089d6 dns_zone_name = oxide.test found_so_far = 1 version = 2
28532024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
28542024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-05-30 04:28:26.957888 UTC dns_group = external generation = 2
28552024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 27.740419ms iteration = 8
28562024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28572024-05-30T04:35:02.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28582024-05-30T04:35:02.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
28592024-05-30T04:35:02.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28602024-05-30T04:35:02.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
28612024-05-30T04:35:02.184ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
28622024-05-30T04:35:02.184ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.026357ms iteration = 15
28632024-05-30T04:35:02.188ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
28642024-05-30T04:35:02.188ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.060195ms iteration = 43
28652024-05-30T04:35:02.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28662024-05-30T04:35:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28672024-05-30T04:35:02.189ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28682024-05-30T04:35:02.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28692024-05-30T04:35:02.190ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28702024-05-30T04:35:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28712024-05-30T04:35:02.191ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28722024-05-30T04:35:02.191ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28732024-05-30T04:35:02.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28742024-05-30T04:35:02.192ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28752024-05-30T04:35:02.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28762024-05-30T04:35:02.193ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28772024-05-30T04:35:02.194ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
28782024-05-30T04:35:02.194ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28792024-05-30T04:35:02.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28802024-05-30T04:35:02.195ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28812024-05-30T04:35:02.195ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28822024-05-30T04:35:02.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28832024-05-30T04:35:02.195ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.838262ms iteration = 15
28842024-05-30T04:35:02.219ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-05-30T04:28:02.352462Z, time_modified: 2024-05-30T04:28:02.352462Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, db_silo: Silo { identity: SiloIdentity { id: d8f69ca2-c269-4e82-bfe6-cba0d46dd6ab, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-05-30T04:28:26.925019Z, time_modified: 2024-05-30T04:28:26.925019Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "3978f4593add79bfb83adcdfaedff6903a142b495fdc18a0ca0f17f448815b03" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
28852024-05-30T04:35:02.219ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 54.529068ms iteration = 8
28862024-05-30T04:35:02.345ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
28872024-05-30T04:35:02.345ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28882024-05-30T04:35:02.346ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "89977e7c-4e56-4991-a27c-1132d9831aee", "content-length": "26", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
28892024-05-30T04:35:02.346ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28902024-05-30T04:35:02.348ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28912024-05-30T04:35:02.359ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
28922024-05-30T04:35:02.375ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
28932024-05-30T04:35:02.376ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e38f6647-9fd8-42e1-a0dd-fe0facd273d2", "content-length": "691", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
28942024-05-30T04:35:02.376ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
28952024-05-30T04:35:02.376ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
28962024-05-30T04:35:02.376ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
28972024-05-30T04:35:02.377ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 210.955327ms iteration = 15
28982024-05-30T04:35:02.395ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
28992024-05-30T04:35:02.407ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
29002024-05-30T04:35:02.408ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29012024-05-30T04:35:02.408ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9f7329f0-89c6-4194-a719-55e3b5b0dd2d", "content-length": "26", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
29022024-05-30T04:35:02.408ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29032024-05-30T04:35:02.409ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29042024-05-30T04:35:02.531ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
29052024-05-30T04:35:02.531ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29062024-05-30T04:35:02.533ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "281be594-175c-4114-acbf-a864d5600fd0", "content-length": "26", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
29072024-05-30T04:35:02.533ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29082024-05-30T04:35:02.534ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
29092024-05-30T04:35:02.551ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
29102024-05-30T04:35:02.552ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "513f2c38-9e93-4349-af30-60d974ee6cdd", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
29112024-05-30T04:35:02.552ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
29122024-05-30T04:35:02.553ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f189427f-dec1-4119-98c6-f7f9903a1569", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
29132024-05-30T04:35:02.553ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
29142024-05-30T04:35:02.554ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
29152024-05-30T04:35:02.554ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 389.123067ms iteration = 10
29162024-05-30T04:35:02.559ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
29172024-05-30T04:35:02.559ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a40dfb7d-7022-4198-81b3-feb337f3bdff", "content-length": "2", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
29182024-05-30T04:35:02.560ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29192024-05-30T04:35:02.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29202024-05-30T04:35:02.562ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 396.664624ms iteration = 15
29212024-05-30T04:35:02.636ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 470.837957ms iteration = 15
29222024-05-30T04:35:02.644ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
29232024-05-30T04:35:02.708ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
29242024-05-30T04:35:02.709ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8a6c83bf-a247-4fd7-983a-ae1bafd332b9", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29252024-05-30T04:35:02.709ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29262024-05-30T04:35:02.711ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5c85d1f7-ddeb-40d9-8539-3f3fd87bbe6e", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29272024-05-30T04:35:02.711ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29282024-05-30T04:35:02.713ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d9fe56c5-1357-4aa9-8444-3dfc7c359a30", "date": "Thu, 30 May 2024 04:35:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29292024-05-30T04:35:02.713ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29302024-05-30T04:35:02.715ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
29312024-05-30T04:35:02.715ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 560.58122ms iteration = 10
29322024-05-30T04:35:02.729ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29332024-05-30T04:35:02.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29342024-05-30T04:35:02.731ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
29352024-05-30T04:35:02.731ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29362024-05-30T04:35:02.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29372024-05-30T04:35:02.738ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
29382024-05-30T04:35:02.738ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6c58a1c2-0000-4731-b2eb-6dc6334e6eff", "content-length": "121", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
29392024-05-30T04:35:02.738ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
29402024-05-30T04:35:02.738ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
29412024-05-30T04:35:02.738ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
29422024-05-30T04:35:02.738ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
29432024-05-30T04:35:02.738ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
29442024-05-30T04:35:02.738ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
29452024-05-30T04:35:02.738ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
29462024-05-30T04:35:02.738ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
29472024-05-30T04:35:02.739ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
29482024-05-30T04:35:02.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29492024-05-30T04:35:02.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29502024-05-30T04:35:02.740ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
29512024-05-30T04:35:02.740ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8be541eb-4f43-4fee-af7e-7f14135861fb", "content-length": "2", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
29522024-05-30T04:35:02.740ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
29532024-05-30T04:35:02.740ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "22d93111-f646-43eb-8c77-c9899e816c8f", "content-length": "52", "date": "Thu, 30 May 2024 04:35:01 GMT"} })
29542024-05-30T04:35:02.740ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
29552024-05-30T04:35:02.740ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29562024-05-30T04:35:02.783ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fcb400c9-a396-48ef-86d8-b5c8a22b2072", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
29572024-05-30T04:35:02.783ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
29582024-05-30T04:35:02.783ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29592024-05-30T04:35:02.784ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f6e34962-0ac8-4129-abd3-d6f323a798d0", "content-length": "467", "date": "Thu, 30 May 2024 04:35:02 GMT"} })
29602024-05-30T04:35:02.784ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29612024-05-30T04:35:02.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29622024-05-30T04:35:02.786ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29632024-05-30T04:35:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29642024-05-30T04:35:02.789ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29652024-05-30T04:35:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29662024-05-30T04:35:02.790ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29672024-05-30T04:35:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29682024-05-30T04:35:02.798ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 631.986131ms iteration = 15
29692024-05-30T04:35:02.870ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 224 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 9305c9d9-13b5-4d00-b40e-a5028043bb6a response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
29702024-05-30T04:35:12.166ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
29712024-05-30T04:35:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29722024-05-30T04:35:12.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29732024-05-30T04:35:12.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29742024-05-30T04:35:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29752024-05-30T04:35:12.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29762024-05-30T04:35:12.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29772024-05-30T04:35:12.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29782024-05-30T04:35:12.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29792024-05-30T04:35:12.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29802024-05-30T04:35:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
29812024-05-30T04:35:12.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29822024-05-30T04:35:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29832024-05-30T04:35:12.179ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
29842024-05-30T04:35:12.179ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.274453ms iteration = 44
29852024-05-30T04:35:12.870ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 136 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 2141f315-bde1-45f5-a6d4-78340b9f477a response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
29862024-05-30T04:35:22.166ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
29872024-05-30T04:35:22.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29882024-05-30T04:35:22.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29892024-05-30T04:35:22.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29902024-05-30T04:35:22.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29912024-05-30T04:35:22.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29922024-05-30T04:35:22.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29932024-05-30T04:35:22.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29942024-05-30T04:35:22.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29952024-05-30T04:35:22.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29962024-05-30T04:35:22.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
29972024-05-30T04:35:22.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29982024-05-30T04:35:22.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29992024-05-30T04:35:22.181ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
30002024-05-30T04:35:22.181ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.974953ms iteration = 45
30012024-05-30T04:35:22.871ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 737 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = 402d3f01-411c-4f4a-926a-9cd134d8de75 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
30022024-05-30T04:35:32.165ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
30032024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
30042024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
30052024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30062024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
30072024-05-30T04:35:32.166ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
30082024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
30092024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
30102024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30112024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30122024-05-30T04:35:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
30132024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
30142024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
30152024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30162024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
30172024-05-30T04:35:32.167ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
30182024-05-30T04:35:32.167ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
30192024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30202024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.13\u{b5}s iteration = 16
30212024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
30222024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
30232024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30242024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30252024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
30262024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30272024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30282024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30292024-05-30T04:35:32.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30302024-05-30T04:35:32.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1354, tv_nsec: 277852544 } })
30312024-05-30T04:35:32.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30322024-05-30T04:35:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30332024-05-30T04:35:32.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30342024-05-30T04:35:32.168ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1354, tv_nsec: 278072396 } })
30352024-05-30T04:35:32.168ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
30362024-05-30T04:35:32.168ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.287666ms iteration = 16
30372024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30382024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30392024-05-30T04:35:32.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1029
30402024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30412024-05-30T04:35:32.169ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1029
30422024-05-30T04:35:32.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30432024-05-30T04:35:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30442024-05-30T04:35:32.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30452024-05-30T04:35:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30462024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30472024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30482024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30492024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30502024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30512024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30522024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30532024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30542024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30552024-05-30T04:35:32.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30562024-05-30T04:35:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30572024-05-30T04:35:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30582024-05-30T04:35:32.174ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30592024-05-30T04:35:32.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30602024-05-30T04:35:32.175ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:100
30612024-05-30T04:35:32.175ZWARNnexus (BackgroundTasks): Physical Disk Adoption: skipped background_task = physical_disk_adoption file = nexus/src/app/background/physical_disk_adoption.rs:104 reason = no inventory
30622024-05-30T04:35:32.175ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 7.641538ms iteration = 16
30632024-05-30T04:35:32.175ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30642024-05-30T04:35:32.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30652024-05-30T04:35:32.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30662024-05-30T04:35:32.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-73ed95a1-f279-488a-9386-6d3e4371db4d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1354, tv_nsec: 285546812 } })
30672024-05-30T04:35:32.175ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1029 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
30682024-05-30T04:35:32.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
30692024-05-30T04:35:32.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30702024-05-30T04:35:32.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30712024-05-30T04:35:32.180ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/instance_watcher.rs:473 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
30722024-05-30T04:35:32.180ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.660425ms iteration = 16
30732024-05-30T04:35:32.183ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
30742024-05-30T04:35:32.183ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.553741ms iteration = 46
30752024-05-30T04:35:32.186ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30762024-05-30T04:35:32.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30772024-05-30T04:35:32.187ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30782024-05-30T04:35:32.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30792024-05-30T04:35:32.187ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30802024-05-30T04:35:32.188ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30812024-05-30T04:35:32.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30822024-05-30T04:35:32.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30832024-05-30T04:35:32.189ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30842024-05-30T04:35:32.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30852024-05-30T04:35:32.189ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30862024-05-30T04:35:32.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30872024-05-30T04:35:32.191ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
30882024-05-30T04:35:32.191ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30892024-05-30T04:35:32.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30902024-05-30T04:35:32.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 73ed95a1-f279-488a-9386-6d3e4371db4d, lookup_type: ById(73ed95a1-f279-488a-9386-6d3e4371db4d) } result = Ok(())
30912024-05-30T04:35:32.192ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30922024-05-30T04:35:32.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30932024-05-30T04:35:32.193ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.03115ms iteration = 16
30942024-05-30T04:35:32.253ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
30952024-05-30T04:35:32.253ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30962024-05-30T04:35:32.255ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0e747ae3-8157-4327-8261-e153dd2bde39", "content-length": "26", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
30972024-05-30T04:35:32.255ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30982024-05-30T04:35:32.256ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30992024-05-30T04:35:32.257ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1037 zone_address = fd00:1122:3344:101::2
31002024-05-30T04:35:32.257ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31012024-05-30T04:35:32.257ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "38ca6bad-167b-47d5-9089-019e10d5b916", "content-length": "26", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
31022024-05-30T04:35:32.257ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1042 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31032024-05-30T04:35:32.259ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
31042024-05-30T04:35:32.276ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1037 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 zone_address = fd00:1122:3344:101::2
31052024-05-30T04:35:32.276ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31062024-05-30T04:35:32.276ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e82f9097-07c4-4817-8462-046ebcc5dad6", "content-length": "26", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
31072024-05-30T04:35:32.276ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1042 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31082024-05-30T04:35:32.278ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1074 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
31092024-05-30T04:35:32.286ZDEBGnexus (BackgroundTasks): client request SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
31102024-05-30T04:35:32.287ZDEBGnexus (BackgroundTasks): client response SledAgent = 73ed95a1-f279-488a-9386-6d3e4371db4d background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e928caa6-f494-4b91-82d3-d07d06b61cc2", "content-length": "691", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
31112024-05-30T04:35:32.287ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:123 interfaces = {} sled = buskin
31122024-05-30T04:35:32.287ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:135 mappings = [] sled = buskin
31132024-05-30T04:35:32.287ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/v2p_mappings.rs:148 mappings = [] sled = buskin
31142024-05-30T04:35:32.289ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 121.64723ms iteration = 16
31152024-05-30T04:35:32.362ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 196.625622ms iteration = 16
31162024-05-30T04:35:32.368ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
31172024-05-30T04:35:32.368ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d5a09ed1-9200-44f8-ace0-82596cd2fe77", "content-length": "2", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
31182024-05-30T04:35:32.370ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
31192024-05-30T04:35:32.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
31202024-05-30T04:35:32.371ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 205.745598ms iteration = 16
31212024-05-30T04:35:32.432ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31222024-05-30T04:35:32.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31232024-05-30T04:35:32.434ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:145 port = SwitchPort { id: 40d3be2a-1ea8-4f90-84d5-9a39405ed8e4, rack_id: 4c05e238-e08c-41b5-985f-5b9151c0e727, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(cc751486-7f5f-4cf7-bb26-9c5ca6fe283e) } rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
31242024-05-30T04:35:32.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31252024-05-30T04:35:32.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31262024-05-30T04:35:32.442ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
31272024-05-30T04:35:32.442ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c6f47390-0b6c-402d-a44d-93b515fe5d78", "content-length": "121", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
31282024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1644 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
31292024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1671 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = cc751486-7f5f-4cf7-bb26-9c5ca6fe283e
31302024-05-30T04:35:32.442ZERROnexus (BackgroundTasks): unable to retrieve routes from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1783 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
31312024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:370 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
31322024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:374 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
31332024-05-30T04:35:32.442ZWARNnexus (BackgroundTasks): no discovered routes from switch. it is possible that an earlier api call failed. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1512 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
31342024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
31352024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:389 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 routes = {}
31362024-05-30T04:35:32.442ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:409 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727
31372024-05-30T04:35:32.442ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31382024-05-30T04:35:32.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31392024-05-30T04:35:32.444ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
31402024-05-30T04:35:32.444ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f21e0bd4-d8d1-4b96-8093-2662a6db7374", "content-length": "2", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
31412024-05-30T04:35:32.444ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
31422024-05-30T04:35:32.444ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "24412f40-6af9-4a5f-9d34-fe972d0424b9", "content-length": "52", "date": "Thu, 30 May 2024 04:35:31 GMT"} })
31432024-05-30T04:35:32.444ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Net { addr: 10.151.1.174, width: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:458 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch_location = Switch0
31442024-05-30T04:35:32.444ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
31452024-05-30T04:35:32.492ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a66e2a90-8f2a-4e1e-9990-322e5da33b01", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
31462024-05-30T04:35:32.492ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:761 rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 switch = Switch0
31472024-05-30T04:35:32.492ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
31482024-05-30T04:35:32.537ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 4c05e238-e08c-41b5-985f-5b9151c0e727 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "32bd56ef-264f-4e82-a3cf-f736d90c2d05", "content-length": "467", "date": "Thu, 30 May 2024 04:35:32 GMT"} })
31492024-05-30T04:35:32.537ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31502024-05-30T04:35:32.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31512024-05-30T04:35:32.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31522024-05-30T04:35:32.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31532024-05-30T04:35:32.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31542024-05-30T04:35:32.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31552024-05-30T04:35:32.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31562024-05-30T04:35:32.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31572024-05-30T04:35:32.549ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 382.041776ms iteration = 16
31582024-05-30T04:35:32.871ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 169 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = cf784c75-0666-4f09-848a-ce46f399ca64 response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a
31592024-05-30T04:35:42.166ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
31602024-05-30T04:35:42.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31612024-05-30T04:35:42.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31622024-05-30T04:35:42.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31632024-05-30T04:35:42.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31642024-05-30T04:35:42.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31652024-05-30T04:35:42.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31662024-05-30T04:35:42.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31672024-05-30T04:35:42.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31682024-05-30T04:35:42.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31692024-05-30T04:35:42.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: b9a32bb6-c543-452c-bbbf-8209a8288553, lookup_type: ById(b9a32bb6-c543-452c-bbbf-8209a8288553) } result = Ok(())
31702024-05-30T04:35:42.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31712024-05-30T04:35:42.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31722024-05-30T04:35:42.181ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 original_time_created = 2024-05-30 04:27:50.708427 UTC target_id = b9a32bb6-c543-452c-bbbf-8209a8288553 time_created = 2024-05-30 04:27:50.708427 UTC
31732024-05-30T04:35:42.181ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.321107ms iteration = 47
31742024-05-30T04:35:42.871ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/0cd0e82/dropshot/src/server.rs:878 latency_us = 136 local_addr = [fd00:1122:3344:101::b]:41031 method = GET remote_addr = [fd00:1122:3344:101::d]:53315 req_id = c97bef9e-c049-4bfb-9469-a978bd0fbeee response_code = 200 uri = /b5bf7e90-33d3-4f67-b9d3-793143654e9a