Max level shown:
12024-04-03T10:15:04.118ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
22024-04-03T10:15:04.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-04-03T10:15:04.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42024-04-03T10:15:04.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52024-04-03T10:15:04.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62024-04-03T10:15:04.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72024-04-03T10:15:04.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
82024-04-03T10:15:04.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92024-04-03T10:15:04.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
102024-04-03T10:15:04.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112024-04-03T10:15:04.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
122024-04-03T10:15:04.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132024-04-03T10:15:04.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142024-04-03T10:15:04.132ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
152024-04-03T10:15:04.132ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.252682ms iteration = 33
162024-04-03T10:15:04.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 290 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = db118106-4a8e-4cb2-9429-22b1dd3227df response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
172024-04-03T10:15:14.118ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
182024-04-03T10:15:14.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192024-04-03T10:15:14.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
202024-04-03T10:15:14.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212024-04-03T10:15:14.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
222024-04-03T10:15:14.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232024-04-03T10:15:14.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
242024-04-03T10:15:14.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
252024-04-03T10:15:14.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
262024-04-03T10:15:14.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
272024-04-03T10:15:14.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
282024-04-03T10:15:14.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
292024-04-03T10:15:14.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
302024-04-03T10:15:14.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
312024-04-03T10:15:14.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.253046ms iteration = 34
322024-04-03T10:15:14.628ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 171 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = e162ca78-d4d6-4a0e-8c54-ae2893ccbbff response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
332024-04-03T10:15:24.079ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
342024-04-03T10:15:24.079ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
352024-04-03T10:15:24.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
362024-04-03T10:15:24.081ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 346396769 } })
372024-04-03T10:15:24.081ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
382024-04-03T10:15:24.112ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
392024-04-03T10:15:24.112ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
402024-04-03T10:15:24.112ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4f2caf71-252e-40ea-89db-5192ace8b3aa", "content-length": "26", "date": "Wed, 03 Apr 2024 10:15:24 GMT"} })
412024-04-03T10:15:24.112ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
422024-04-03T10:15:24.113ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
432024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
442024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
452024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
462024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
472024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
482024-04-03T10:15:24.118ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
492024-04-03T10:15:24.118ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
502024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.01\u{b5}s iteration = 13
512024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
522024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
532024-04-03T10:15:24.118ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
542024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
552024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
562024-04-03T10:15:24.118ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
572024-04-03T10:15:24.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
582024-04-03T10:15:24.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
592024-04-03T10:15:24.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
602024-04-03T10:15:24.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
612024-04-03T10:15:24.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
622024-04-03T10:15:24.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
632024-04-03T10:15:24.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
642024-04-03T10:15:24.121ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
652024-04-03T10:15:24.121ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.475295ms iteration = 13
662024-04-03T10:15:24.121ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 386187796 } })
672024-04-03T10:15:24.121ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
682024-04-03T10:15:24.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
692024-04-03T10:15:24.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
702024-04-03T10:15:24.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
712024-04-03T10:15:24.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
722024-04-03T10:15:24.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
732024-04-03T10:15:24.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
742024-04-03T10:15:24.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
752024-04-03T10:15:24.124ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
762024-04-03T10:15:24.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
772024-04-03T10:15:24.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
782024-04-03T10:15:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
792024-04-03T10:15:24.125ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 390763809 } })
802024-04-03T10:15:24.125ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
812024-04-03T10:15:24.126ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
822024-04-03T10:15:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
832024-04-03T10:15:24.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
842024-04-03T10:15:24.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
852024-04-03T10:15:24.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
862024-04-03T10:15:24.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
872024-04-03T10:15:24.133ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
882024-04-03T10:15:24.133ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.235959ms iteration = 35
892024-04-03T10:15:24.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
902024-04-03T10:15:24.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
912024-04-03T10:15:24.142ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
922024-04-03T10:15:24.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
932024-04-03T10:15:24.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
942024-04-03T10:15:24.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
952024-04-03T10:15:24.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
962024-04-03T10:15:24.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
972024-04-03T10:15:24.148ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 29.760513ms iteration = 13
982024-04-03T10:15:24.191ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 112.735349ms iteration = 13
992024-04-03T10:15:24.196ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
1002024-04-03T10:15:24.196ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1012024-04-03T10:15:24.196ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
1022024-04-03T10:15:24.196ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1032024-04-03T10:15:24.196ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7b275b6b-12bb-409a-99e5-ae2b313a56f9", "content-length": "26", "date": "Wed, 03 Apr 2024 10:15:24 GMT"} })
1042024-04-03T10:15:24.196ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1052024-04-03T10:15:24.197ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "203fef15-8c4d-4115-a988-06fe1c700eed", "content-length": "26", "date": "Wed, 03 Apr 2024 10:15:24 GMT"} })
1062024-04-03T10:15:24.197ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1072024-04-03T10:15:24.197ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1082024-04-03T10:15:24.198ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
1092024-04-03T10:15:24.269ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1102024-04-03T10:15:24.269ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3abd90a6-a508-4bba-a152-5c2a8f4ce37d", "content-length": "1", "date": "Wed, 03 Apr 2024 10:15:23 GMT"} })
1112024-04-03T10:15:24.271ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1122024-04-03T10:15:24.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1132024-04-03T10:15:24.272ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 154.208345ms iteration = 13
1142024-04-03T10:15:24.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1152024-04-03T10:15:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1162024-04-03T10:15:24.326ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
1172024-04-03T10:15:24.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1182024-04-03T10:15:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1192024-04-03T10:15:24.334ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1202024-04-03T10:15:24.334ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "60dc6d13-41a5-40e8-a317-215441654415", "content-length": "121", "date": "Wed, 03 Apr 2024 10:15:23 GMT"} })
1212024-04-03T10:15:24.334ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
1222024-04-03T10:15:24.334ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
1232024-04-03T10:15:24.335ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
1242024-04-03T10:15:24.335ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
1252024-04-03T10:15:24.335ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
1262024-04-03T10:15:24.335ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
1272024-04-03T10:15:24.335ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
1282024-04-03T10:15:24.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1292024-04-03T10:15:24.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1302024-04-03T10:15:24.337ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
1312024-04-03T10:15:24.337ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a65bc089-f16b-4186-aef9-e54ee400381c", "content-length": "2", "date": "Wed, 03 Apr 2024 10:15:23 GMT"} })
1322024-04-03T10:15:24.337ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
1332024-04-03T10:15:24.338ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "99224c36-b505-4e6f-97d8-16373ba7e810", "content-length": "52", "date": "Wed, 03 Apr 2024 10:15:23 GMT"} })
1342024-04-03T10:15:24.338ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
1352024-04-03T10:15:24.338ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
1362024-04-03T10:15:24.385ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f08ce1ff-5b9d-4323-aed2-d4cdeac0eb52", "date": "Wed, 03 Apr 2024 10:15:24 GMT"} })
1372024-04-03T10:15:24.386ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
1382024-04-03T10:15:24.386ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
1392024-04-03T10:15:24.389ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5fb50b33-6e46-4bbf-9695-749af59625a0", "content-length": "452", "date": "Wed, 03 Apr 2024 10:15:24 GMT"} })
1402024-04-03T10:15:24.389ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1412024-04-03T10:15:24.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1422024-04-03T10:15:24.392ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1432024-04-03T10:15:24.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1442024-04-03T10:15:24.394ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1452024-04-03T10:15:24.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1462024-04-03T10:15:24.396ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1472024-04-03T10:15:24.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1482024-04-03T10:15:24.403ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 284.398483ms iteration = 13
1492024-04-03T10:15:24.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 938 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 6011b24a-bb75-4449-9b5c-ae1f90492be0 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
1502024-04-03T10:15:34.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
1512024-04-03T10:15:34.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1522024-04-03T10:15:34.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1532024-04-03T10:15:34.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1542024-04-03T10:15:34.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1552024-04-03T10:15:34.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1562024-04-03T10:15:34.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1572024-04-03T10:15:34.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1582024-04-03T10:15:34.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1592024-04-03T10:15:34.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602024-04-03T10:15:34.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
1612024-04-03T10:15:34.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1622024-04-03T10:15:34.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1632024-04-03T10:15:34.133ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
1642024-04-03T10:15:34.133ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.810982ms iteration = 36
1652024-04-03T10:15:34.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 172 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = ed22b3ab-af9e-4863-9e92-c128f3ba459a response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
1662024-04-03T10:15:44.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
1672024-04-03T10:15:44.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1682024-04-03T10:15:44.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1692024-04-03T10:15:44.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1702024-04-03T10:15:44.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1712024-04-03T10:15:44.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1722024-04-03T10:15:44.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1732024-04-03T10:15:44.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1742024-04-03T10:15:44.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1752024-04-03T10:15:44.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1762024-04-03T10:15:44.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
1772024-04-03T10:15:44.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1782024-04-03T10:15:44.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1792024-04-03T10:15:44.134ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
1802024-04-03T10:15:44.134ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.154357ms iteration = 37
1812024-04-03T10:15:44.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 190 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 4d438638-1187-4847-bb87-7c3a766be06f response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
1822024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
1832024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
1842024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1852024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1862024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
1872024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
1882024-04-03T10:15:54.078ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1892024-04-03T10:15:54.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
1902024-04-03T10:15:54.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1912024-04-03T10:15:54.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1922024-04-03T10:15:54.081ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 345974652 } })
1932024-04-03T10:15:54.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
1942024-04-03T10:15:54.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1952024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
1962024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
1972024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
1982024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
1992024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
2002024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2012024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
2022024-04-03T10:15:54.084ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
2032024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.9\u{b5}s iteration = 8
2042024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
2052024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2062024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2072024-04-03T10:15:54.084ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
2082024-04-03T10:15:54.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2092024-04-03T10:15:54.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2102024-04-03T10:15:54.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2112024-04-03T10:15:54.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2122024-04-03T10:15:54.086ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2fbdcbcd-ff1b-4375-856b-f7f3530b0e63.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 350689539 } })
2132024-04-03T10:15:54.086ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e04e52c-490d-4b79-9cc6-e5adae4e5501.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("586c70fe-6c95-438f-be65-3daafd7be41b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 350774880 } })
2142024-04-03T10:15:54.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2152024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
2162024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2172024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
2182024-04-03T10:15:54.087ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27
2192024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
2202024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.961\u{b5}s iteration = 9
2212024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2222024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
2232024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.578107ms iteration = 8
2242024-04-03T10:15:54.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2252024-04-03T10:15:54.088ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
2262024-04-03T10:15:54.088ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.748649ms iteration = 8
2272024-04-03T10:15:54.088ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2282024-04-03T10:15:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2292024-04-03T10:15:54.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2302024-04-03T10:15:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2312024-04-03T10:15:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2322024-04-03T10:15:54.091ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2332024-04-03T10:15:54.091ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2342024-04-03T10:15:54.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2352024-04-03T10:15:54.092ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
2362024-04-03T10:15:54.092ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
2372024-04-03T10:15:54.092ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
2382024-04-03T10:15:54.092ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2392024-04-03T10:15:54.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2402024-04-03T10:15:54.093ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2412024-04-03T10:15:54.093ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2422024-04-03T10:15:54.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2432024-04-03T10:15:54.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2442024-04-03T10:15:54.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2452024-04-03T10:15:54.094ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2462024-04-03T10:15:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2472024-04-03T10:15:54.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2482024-04-03T10:15:54.095ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2492024-04-03T10:15:54.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2502024-04-03T10:15:54.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2512024-04-03T10:15:54.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2522024-04-03T10:15:54.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2532024-04-03T10:15:54.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2542024-04-03T10:15:54.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2552024-04-03T10:15:54.100ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2562024-04-03T10:15:54.100ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 0 version = 2
2572024-04-03T10:15:54.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2582024-04-03T10:15:54.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2592024-04-03T10:15:54.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
2602024-04-03T10:15:54.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2612024-04-03T10:15:54.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2622024-04-03T10:15:54.101ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2632024-04-03T10:15:54.102ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
2642024-04-03T10:15:54.102ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2652024-04-03T10:15:54.102ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-04-03 10:10:24.516388 UTC dns_group = internal generation = 1
2662024-04-03T10:15:54.102ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.017888ms iteration = 8
2672024-04-03T10:15:54.102ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2682024-04-03T10:15:54.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2692024-04-03T10:15:54.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
2702024-04-03T10:15:54.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2712024-04-03T10:15:54.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2722024-04-03T10:15:54.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2732024-04-03T10:15:54.106ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 1 version = 2
2742024-04-03T10:15:54.106ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
2752024-04-03T10:15:54.106ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-04-03 10:10:25.138074 UTC dns_group = external generation = 2
2762024-04-03T10:15:54.106ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 21.575454ms iteration = 8
2772024-04-03T10:15:54.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
2782024-04-03T10:15:54.107ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2792024-04-03T10:15:54.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
2802024-04-03T10:15:54.115ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-04-03T10:09:54.267934Z, time_modified: 2024-04-03T10:09:54.267934Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }) }
2812024-04-03T10:15:54.115ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 28.698373ms iteration = 8
2822024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
2832024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
2842024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2852024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2862024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
2872024-04-03T10:15:54.119ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
2882024-04-03T10:15:54.119ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
2892024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 5.78\u{b5}s iteration = 14
2902024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
2912024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
2922024-04-03T10:15:54.119ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
2932024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2942024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
2952024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2962024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2972024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
2982024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2992024-04-03T10:15:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3002024-04-03T10:15:54.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012024-04-03T10:15:54.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3022024-04-03T10:15:54.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3032024-04-03T10:15:54.121ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 386283968 } })
3042024-04-03T10:15:54.122ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
3052024-04-03T10:15:54.122ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.26575ms iteration = 14
3062024-04-03T10:15:54.122ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
3072024-04-03T10:15:54.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3082024-04-03T10:15:54.123ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3092024-04-03T10:15:54.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3102024-04-03T10:15:54.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
3112024-04-03T10:15:54.124ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3122024-04-03T10:15:54.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3132024-04-03T10:15:54.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3142024-04-03T10:15:54.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3152024-04-03T10:15:54.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3162024-04-03T10:15:54.125ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 390269741 } })
3172024-04-03T10:15:54.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3182024-04-03T10:15:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3192024-04-03T10:15:54.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3202024-04-03T10:15:54.126ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
3212024-04-03T10:15:54.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3222024-04-03T10:15:54.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3232024-04-03T10:15:54.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
3242024-04-03T10:15:54.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3252024-04-03T10:15:54.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3262024-04-03T10:15:54.134ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
3272024-04-03T10:15:54.134ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.606985ms iteration = 38
3282024-04-03T10:15:54.137ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3292024-04-03T10:15:54.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3302024-04-03T10:15:54.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3312024-04-03T10:15:54.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3322024-04-03T10:15:54.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3332024-04-03T10:15:54.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
3342024-04-03T10:15:54.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3352024-04-03T10:15:54.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3362024-04-03T10:15:54.144ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.128319ms iteration = 14
3372024-04-03T10:15:54.204ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
3382024-04-03T10:15:54.204ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3392024-04-03T10:15:54.206ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "87dc7e2b-1b5a-4dae-9c2e-41733eea4f5f", "content-length": "26", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} })
3402024-04-03T10:15:54.206ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3412024-04-03T10:15:54.207ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3422024-04-03T10:15:54.214ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3432024-04-03T10:15:54.287ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
3442024-04-03T10:15:54.287ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3452024-04-03T10:15:54.288ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3d8e78c1-a302-4742-9a13-9278d7e6fe63", "content-length": "26", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} })
3462024-04-03T10:15:54.288ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3472024-04-03T10:15:54.290ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3482024-04-03T10:15:54.311ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3492024-04-03T10:15:54.355ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3502024-04-03T10:15:54.357ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "95635edc-3492-4735-9b0f-3acb1b8424b2", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3512024-04-03T10:15:54.357ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3522024-04-03T10:15:54.359ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "1888c30d-ce11-4c0e-b34f-ac5463288e67", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3532024-04-03T10:15:54.359ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3542024-04-03T10:15:54.360ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3552024-04-03T10:15:54.360ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 273.3135ms iteration = 10
3562024-04-03T10:15:54.361ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 282.432586ms iteration = 14
3572024-04-03T10:15:54.411ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
3582024-04-03T10:15:54.411ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cba6869c-d7bf-4602-8099-cd66e7781c0a", "content-length": "1", "date": "Wed, 03 Apr 2024 10:15:53 GMT"} })
3592024-04-03T10:15:54.412ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
3602024-04-03T10:15:54.412ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3612024-04-03T10:15:54.412ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4a3c912c-a49f-4a6b-b76d-c71db3b8b1eb", "content-length": "26", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} })
3622024-04-03T10:15:54.412ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3632024-04-03T10:15:54.414ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
3642024-04-03T10:15:54.414ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
3652024-04-03T10:15:54.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3662024-04-03T10:15:54.416ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 297.305339ms iteration = 14
3672024-04-03T10:15:54.444ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3682024-04-03T10:15:54.502ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3692024-04-03T10:15:54.503ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ce9ab278-4002-4fdb-b060-d12362353ded", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3702024-04-03T10:15:54.503ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3712024-04-03T10:15:54.504ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "972d0e20-351d-44c7-9a31-932a4358889b", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3722024-04-03T10:15:54.504ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3732024-04-03T10:15:54.506ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "73304775-8655-433c-8573-6e460f60b8b9", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3742024-04-03T10:15:54.506ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3752024-04-03T10:15:54.508ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3762024-04-03T10:15:54.508ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 423.998801ms iteration = 10
3772024-04-03T10:15:54.548ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3782024-04-03T10:15:54.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3792024-04-03T10:15:54.549ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
3802024-04-03T10:15:54.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3812024-04-03T10:15:54.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3822024-04-03T10:15:54.557ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
3832024-04-03T10:15:54.558ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "76360377-7289-417b-85ee-dbb25e58ee27", "content-length": "121", "date": "Wed, 03 Apr 2024 10:15:53 GMT"} })
3842024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
3852024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
3862024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
3872024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
3882024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
3892024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
3902024-04-03T10:15:54.558ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
3912024-04-03T10:15:54.558ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3922024-04-03T10:15:54.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3932024-04-03T10:15:54.560ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
3942024-04-03T10:15:54.560ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6cd83056-e526-42e8-ace0-dfa337b2448d", "content-length": "2", "date": "Wed, 03 Apr 2024 10:15:53 GMT"} })
3952024-04-03T10:15:54.560ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
3962024-04-03T10:15:54.560ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "122c9e83-ae41-4293-8037-6d6a1ed027eb", "content-length": "52", "date": "Wed, 03 Apr 2024 10:15:53 GMT"} })
3972024-04-03T10:15:54.560ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
3982024-04-03T10:15:54.560ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
3992024-04-03T10:15:54.629ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 195 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = ac57ad40-6334-41a9-9587-547572286640 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
4002024-04-03T10:15:54.643ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8b4a6113-589b-422f-8592-b63f23ea1dd0", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} })
4012024-04-03T10:15:54.643ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
4022024-04-03T10:15:54.643ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4032024-04-03T10:15:54.644ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b779e39d-9eaa-47a2-9830-f9632add039f", "content-length": "452", "date": "Wed, 03 Apr 2024 10:15:54 GMT"} })
4042024-04-03T10:15:54.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4052024-04-03T10:15:54.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4062024-04-03T10:15:54.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4072024-04-03T10:15:54.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4082024-04-03T10:15:54.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4092024-04-03T10:15:54.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4102024-04-03T10:15:54.650ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4112024-04-03T10:15:54.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4122024-04-03T10:15:54.657ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 538.38227ms iteration = 14
4132024-04-03T10:16:04.118ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
4142024-04-03T10:16:04.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4152024-04-03T10:16:04.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4162024-04-03T10:16:04.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4172024-04-03T10:16:04.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4182024-04-03T10:16:04.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4192024-04-03T10:16:04.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4202024-04-03T10:16:04.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4212024-04-03T10:16:04.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4222024-04-03T10:16:04.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4232024-04-03T10:16:04.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
4242024-04-03T10:16:04.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4252024-04-03T10:16:04.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4262024-04-03T10:16:04.133ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
4272024-04-03T10:16:04.134ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.202387ms iteration = 39
4282024-04-03T10:16:04.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 202 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 94d92370-0e8a-4f29-b82f-74ca0dfa3d61 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
4292024-04-03T10:16:14.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
4302024-04-03T10:16:14.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4312024-04-03T10:16:14.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4322024-04-03T10:16:14.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4332024-04-03T10:16:14.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4342024-04-03T10:16:14.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4352024-04-03T10:16:14.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4362024-04-03T10:16:14.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4372024-04-03T10:16:14.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4382024-04-03T10:16:14.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4392024-04-03T10:16:14.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
4402024-04-03T10:16:14.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4412024-04-03T10:16:14.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4422024-04-03T10:16:14.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
4432024-04-03T10:16:14.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.549662ms iteration = 40
4442024-04-03T10:16:14.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 299 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = f85685c4-36bf-48a2-9fee-e7c8d1916a2a response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
4452024-04-03T10:16:21.055ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:765 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1:0]:43836
4462024-04-03T10:16:21.055ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:43836 req_id = b5d3877f-ea23-41a1-b2df-1bdcfb3bc7fb roles = RoleSet { roles: {} } uri = /metrics/producers
4472024-04-03T10:16:21.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:43836 req_id = b5d3877f-ea23-41a1-b2df-1bdcfb3bc7fb resource = Database result = Ok(()) uri = /metrics/producers
4482024-04-03T10:16:21.089ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = eff9cae2-d3de-4f90-b501-6553b5ad9c11
4492024-04-03T10:16:21.089ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:43836 req_id = b5d3877f-ea23-41a1-b2df-1bdcfb3bc7fb roles = RoleSet { roles: {} } uri = /metrics/producers
4502024-04-03T10:16:21.089ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:43836 req_id = b5d3877f-ea23-41a1-b2df-1bdcfb3bc7fb resource = Database result = Ok(()) uri = /metrics/producers
4512024-04-03T10:16:21.097ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = eff9cae2-d3de-4f90-b501-6553b5ad9c11 uri = http://[fd00:1122:3344:101::d]:12223/producers
4522024-04-03T10:16:21.098ZDEBGnexus: client response oximeter-collector = eff9cae2-d3de-4f90-b501-6553b5ad9c11 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "24448bab-6b16-471e-b671-20bce3b2ae2e", "date": "Wed, 03 Apr 2024 10:16:21 GMT"} })
4532024-04-03T10:16:21.098ZINFOnexus: assigned collector to new producer collector_id = eff9cae2-d3de-4f90-b501-6553b5ad9c11 file = nexus/src/app/oximeter.rs:156 producer_id = dcfb65b0-3e90-4921-8143-9084736c941c
4542024-04-03T10:16:21.100ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 44429 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:43836 req_id = b5d3877f-ea23-41a1-b2df-1bdcfb3bc7fb response_code = 204 uri = /metrics/producers
4552024-04-03T10:16:24.079ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
4562024-04-03T10:16:24.079ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4572024-04-03T10:16:24.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4582024-04-03T10:16:24.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 346719479 } })
4592024-04-03T10:16:24.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
4602024-04-03T10:16:24.115ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
4612024-04-03T10:16:24.115ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4622024-04-03T10:16:24.115ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c0b409fb-5368-4d63-a01c-93d79720a2da", "content-length": "26", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
4632024-04-03T10:16:24.115ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4642024-04-03T10:16:24.116ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4652024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
4662024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
4672024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4682024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4692024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
4702024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
4712024-04-03T10:16:24.119ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
4722024-04-03T10:16:24.119ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
4732024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4742024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.41\u{b5}s iteration = 15
4752024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
4762024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4772024-04-03T10:16:24.119ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
4782024-04-03T10:16:24.119ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
4792024-04-03T10:16:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4802024-04-03T10:16:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4812024-04-03T10:16:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4822024-04-03T10:16:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4832024-04-03T10:16:24.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4842024-04-03T10:16:24.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4852024-04-03T10:16:24.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4862024-04-03T10:16:24.122ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 386495255 } })
4872024-04-03T10:16:24.122ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
4882024-04-03T10:16:24.122ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.661609ms iteration = 15
4892024-04-03T10:16:24.123ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
4902024-04-03T10:16:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4912024-04-03T10:16:24.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4922024-04-03T10:16:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4932024-04-03T10:16:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4942024-04-03T10:16:24.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4952024-04-03T10:16:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4962024-04-03T10:16:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4972024-04-03T10:16:24.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4982024-04-03T10:16:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4992024-04-03T10:16:24.127ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 391478506 } })
5002024-04-03T10:16:24.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5012024-04-03T10:16:24.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5022024-04-03T10:16:24.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5032024-04-03T10:16:24.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5042024-04-03T10:16:24.128ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
5052024-04-03T10:16:24.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5062024-04-03T10:16:24.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
5072024-04-03T10:16:24.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5082024-04-03T10:16:24.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5092024-04-03T10:16:24.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
5102024-04-03T10:16:24.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.859409ms iteration = 41
5112024-04-03T10:16:24.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5122024-04-03T10:16:24.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5132024-04-03T10:16:24.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5142024-04-03T10:16:24.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5152024-04-03T10:16:24.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5162024-04-03T10:16:24.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
5172024-04-03T10:16:24.145ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5182024-04-03T10:16:24.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5192024-04-03T10:16:24.146ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.027064ms iteration = 15
5202024-04-03T10:16:24.183ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 103.813585ms iteration = 15
5212024-04-03T10:16:24.196ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
5222024-04-03T10:16:24.196ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5232024-04-03T10:16:24.196ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9e7547aa-7bca-47a4-894b-43626da46cd6", "content-length": "26", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5242024-04-03T10:16:24.196ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5252024-04-03T10:16:24.198ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
5262024-04-03T10:16:24.198ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5272024-04-03T10:16:24.198ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4d324099-311a-4cb2-919f-30eaf15ffb94", "content-length": "26", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5282024-04-03T10:16:24.198ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5292024-04-03T10:16:24.198ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5302024-04-03T10:16:24.200ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
5312024-04-03T10:16:24.269ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5322024-04-03T10:16:24.270ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "dedbc1d0-5333-4e02-869b-3e53c4d62fc9", "content-length": "2", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5332024-04-03T10:16:24.272ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5342024-04-03T10:16:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5352024-04-03T10:16:24.273ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 154.148375ms iteration = 15
5362024-04-03T10:16:24.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5372024-04-03T10:16:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5382024-04-03T10:16:24.326ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
5392024-04-03T10:16:24.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5402024-04-03T10:16:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5412024-04-03T10:16:24.336ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
5422024-04-03T10:16:24.336ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "95970ecb-3302-430f-8212-f1cccb6177dc", "content-length": "121", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5432024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
5442024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
5452024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
5462024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
5472024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
5482024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
5492024-04-03T10:16:24.336ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
5502024-04-03T10:16:24.336ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5512024-04-03T10:16:24.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5522024-04-03T10:16:24.338ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
5532024-04-03T10:16:24.338ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2a82a45a-8c00-4a90-99c7-efddb6b53477", "content-length": "2", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5542024-04-03T10:16:24.338ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
5552024-04-03T10:16:24.339ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "44c60b6b-fd0d-48a3-a812-47b4cb2e735a", "content-length": "52", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5562024-04-03T10:16:24.339ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
5572024-04-03T10:16:24.339ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
5582024-04-03T10:16:24.387ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9eda07c6-bbf5-48b0-a7fa-6cc5d436af05", "date": "Wed, 03 Apr 2024 10:16:23 GMT"} })
5592024-04-03T10:16:24.387ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
5602024-04-03T10:16:24.387ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
5612024-04-03T10:16:24.439ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d2b29d18-4d89-4b58-8a97-8d82c1704c3e", "content-length": "452", "date": "Wed, 03 Apr 2024 10:16:24 GMT"} })
5622024-04-03T10:16:24.439ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5632024-04-03T10:16:24.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5642024-04-03T10:16:24.443ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5652024-04-03T10:16:24.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5662024-04-03T10:16:24.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5672024-04-03T10:16:24.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5682024-04-03T10:16:24.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5692024-04-03T10:16:24.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5702024-04-03T10:16:24.453ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 334.044589ms iteration = 15
5712024-04-03T10:16:24.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 182 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 93f18a2b-a866-42ce-aecf-07044b97d8df response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
5722024-04-03T10:16:34.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
5732024-04-03T10:16:34.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5742024-04-03T10:16:34.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5752024-04-03T10:16:34.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5762024-04-03T10:16:34.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5772024-04-03T10:16:34.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5782024-04-03T10:16:34.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5792024-04-03T10:16:34.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5802024-04-03T10:16:34.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5812024-04-03T10:16:34.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5822024-04-03T10:16:34.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
5832024-04-03T10:16:34.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5842024-04-03T10:16:34.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5852024-04-03T10:16:34.136ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
5862024-04-03T10:16:34.136ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.983601ms iteration = 42
5872024-04-03T10:16:34.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 215 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 7138675f-d817-4b60-80a1-55190989385a response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
5882024-04-03T10:16:44.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
5892024-04-03T10:16:44.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5902024-04-03T10:16:44.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5912024-04-03T10:16:44.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5922024-04-03T10:16:44.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5932024-04-03T10:16:44.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5942024-04-03T10:16:44.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5952024-04-03T10:16:44.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5962024-04-03T10:16:44.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5972024-04-03T10:16:44.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982024-04-03T10:16:44.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
5992024-04-03T10:16:44.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6002024-04-03T10:16:44.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6012024-04-03T10:16:44.134ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
6022024-04-03T10:16:44.134ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.81523ms iteration = 43
6032024-04-03T10:16:44.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 294 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 9336fb84-7ccd-4925-ae05-d1d867950a1b response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
6042024-04-03T10:16:54.078ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
6052024-04-03T10:16:54.078ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6062024-04-03T10:16:54.078ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6072024-04-03T10:16:54.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6082024-04-03T10:16:54.079ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6092024-04-03T10:16:54.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6102024-04-03T10:16:54.079ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
6112024-04-03T10:16:54.079ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6122024-04-03T10:16:54.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6132024-04-03T10:16:54.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6142024-04-03T10:16:54.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 346008480 } })
6152024-04-03T10:16:54.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
6162024-04-03T10:16:54.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6172024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6182024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
6192024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6202024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6212024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6222024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
6232024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
6242024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6252024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6262024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
6272024-04-03T10:16:54.085ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
6282024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.09\u{b5}s iteration = 9
6292024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6302024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
6312024-04-03T10:16:54.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6322024-04-03T10:16:54.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6332024-04-03T10:16:54.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6342024-04-03T10:16:54.087ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2fbdcbcd-ff1b-4375-856b-f7f3530b0e63.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 350929140 } })
6352024-04-03T10:16:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6362024-04-03T10:16:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6372024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
6382024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
6392024-04-03T10:16:54.088ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27
6402024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 8.18\u{b5}s iteration = 10
6412024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
6422024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6432024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6442024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
6452024-04-03T10:16:54.088ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.24726ms iteration = 9
6462024-04-03T10:16:54.088ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e04e52c-490d-4b79-9cc6-e5adae4e5501.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("586c70fe-6c95-438f-be65-3daafd7be41b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 352784154 } })
6472024-04-03T10:16:54.089ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6482024-04-03T10:16:54.091ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
6492024-04-03T10:16:54.091ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.899488ms iteration = 9
6502024-04-03T10:16:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6512024-04-03T10:16:54.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6522024-04-03T10:16:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6532024-04-03T10:16:54.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6542024-04-03T10:16:54.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6552024-04-03T10:16:54.093ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6562024-04-03T10:16:54.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6572024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6582024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6592024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6602024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6612024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
6622024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
6632024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
6642024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6652024-04-03T10:16:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6662024-04-03T10:16:54.095ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6672024-04-03T10:16:54.096ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6682024-04-03T10:16:54.096ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6692024-04-03T10:16:54.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6702024-04-03T10:16:54.097ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6712024-04-03T10:16:54.099ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6722024-04-03T10:16:54.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6732024-04-03T10:16:54.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6742024-04-03T10:16:54.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
6752024-04-03T10:16:54.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6762024-04-03T10:16:54.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6772024-04-03T10:16:54.101ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6782024-04-03T10:16:54.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6792024-04-03T10:16:54.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6802024-04-03T10:16:54.102ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
6812024-04-03T10:16:54.102ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6822024-04-03T10:16:54.102ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-04-03 10:10:24.516388 UTC dns_group = internal generation = 1
6832024-04-03T10:16:54.102ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.560479ms iteration = 9
6842024-04-03T10:16:54.103ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6852024-04-03T10:16:54.103ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6862024-04-03T10:16:54.103ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 0 version = 2
6872024-04-03T10:16:54.103ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6882024-04-03T10:16:54.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6892024-04-03T10:16:54.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6902024-04-03T10:16:54.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6912024-04-03T10:16:54.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6922024-04-03T10:16:54.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6932024-04-03T10:16:54.107ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6942024-04-03T10:16:54.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6952024-04-03T10:16:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6962024-04-03T10:16:54.109ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6972024-04-03T10:16:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6982024-04-03T10:16:54.111ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 1 version = 2
6992024-04-03T10:16:54.111ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
7002024-04-03T10:16:54.111ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-04-03 10:10:25.138074 UTC dns_group = external generation = 2
7012024-04-03T10:16:54.111ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 25.714959ms iteration = 9
7022024-04-03T10:16:54.113ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-04-03T10:09:54.267934Z, time_modified: 2024-04-03T10:09:54.267934Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }) } file = nexus/src/app/background/external_endpoints.rs:101
7032024-04-03T10:16:54.113ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 24.957536ms iteration = 9
7042024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
7052024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
7062024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7072024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
7082024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7092024-04-03T10:16:54.119ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
7102024-04-03T10:16:54.119ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
7112024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.72\u{b5}s iteration = 16
7122024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
7132024-04-03T10:16:54.119ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
7142024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 16 reason = Timeout
7152024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7162024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
7172024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7182024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7192024-04-03T10:16:54.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7202024-04-03T10:16:54.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7212024-04-03T10:16:54.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7222024-04-03T10:16:54.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7232024-04-03T10:16:54.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7242024-04-03T10:16:54.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7252024-04-03T10:16:54.121ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 385753276 } })
7262024-04-03T10:16:54.122ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
7272024-04-03T10:16:54.122ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.542796ms iteration = 16
7282024-04-03T10:16:54.122ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
7292024-04-03T10:16:54.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7302024-04-03T10:16:54.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7312024-04-03T10:16:54.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7322024-04-03T10:16:54.125ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 389896321 } })
7332024-04-03T10:16:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7342024-04-03T10:16:54.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7352024-04-03T10:16:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7362024-04-03T10:16:54.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7372024-04-03T10:16:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7382024-04-03T10:16:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7392024-04-03T10:16:54.127ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
7402024-04-03T10:16:54.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7412024-04-03T10:16:54.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7422024-04-03T10:16:54.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7432024-04-03T10:16:54.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7442024-04-03T10:16:54.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7452024-04-03T10:16:54.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
7462024-04-03T10:16:54.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7472024-04-03T10:16:54.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7482024-04-03T10:16:54.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
7492024-04-03T10:16:54.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.025583ms iteration = 44
7502024-04-03T10:16:54.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7512024-04-03T10:16:54.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7522024-04-03T10:16:54.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7532024-04-03T10:16:54.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7542024-04-03T10:16:54.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7552024-04-03T10:16:54.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
7562024-04-03T10:16:54.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7572024-04-03T10:16:54.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7582024-04-03T10:16:54.144ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 25.060057ms iteration = 16
7592024-04-03T10:16:54.223ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
7602024-04-03T10:16:54.223ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
7612024-04-03T10:16:54.266ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
7622024-04-03T10:16:54.266ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7632024-04-03T10:16:54.266ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2cd95886-35ec-489b-9f13-dc7b6205efd2", "content-length": "26", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
7642024-04-03T10:16:54.266ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7652024-04-03T10:16:54.267ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7662024-04-03T10:16:54.273ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
7672024-04-03T10:16:54.273ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7682024-04-03T10:16:54.273ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3f8c275a-615c-4b05-9b5f-fbff3bb7dc46", "content-length": "26", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
7692024-04-03T10:16:54.273ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7702024-04-03T10:16:54.275ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
7712024-04-03T10:16:54.359ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
7722024-04-03T10:16:54.359ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
7732024-04-03T10:16:54.359ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0f89e043-c8aa-4aa6-b087-5cc0b2ad2c27", "content-length": "26", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
7742024-04-03T10:16:54.359ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
7752024-04-03T10:16:54.361ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
7762024-04-03T10:16:54.368ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7772024-04-03T10:16:54.368ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7782024-04-03T10:16:54.369ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "65166567-352a-434e-b94d-808339c1fd2e", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7792024-04-03T10:16:54.369ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7802024-04-03T10:16:54.370ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fa77ba8c-95f3-42af-a63d-24d14f659756", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7812024-04-03T10:16:54.370ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7822024-04-03T10:16:54.372ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7832024-04-03T10:16:54.372ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 284.627016ms iteration = 11
7842024-04-03T10:16:54.393ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 313.408942ms iteration = 16
7852024-04-03T10:16:54.393ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7862024-04-03T10:16:54.394ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9c6c85eb-50fc-4f24-b164-23a437b1ec99", "content-length": "2", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
7872024-04-03T10:16:54.396ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7882024-04-03T10:16:54.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7892024-04-03T10:16:54.397ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 278.113428ms iteration = 16
7902024-04-03T10:16:54.431ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7912024-04-03T10:16:54.431ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5afd864c-09ce-4f26-989c-e2b4be31251f", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7922024-04-03T10:16:54.431ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7932024-04-03T10:16:54.432ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4a4a1503-7688-44e1-8859-08f4f9e42a3a", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7942024-04-03T10:16:54.432ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7952024-04-03T10:16:54.434ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "51ed380c-6bd5-47ac-a421-4e95e2183b34", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7962024-04-03T10:16:54.434ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7972024-04-03T10:16:54.436ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7982024-04-03T10:16:54.436ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 350.744174ms iteration = 11
7992024-04-03T10:16:54.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8002024-04-03T10:16:54.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8012024-04-03T10:16:54.484ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
8022024-04-03T10:16:54.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8032024-04-03T10:16:54.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8042024-04-03T10:16:54.491ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8052024-04-03T10:16:54.492ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "85238285-b935-4592-8b4d-8e8507961777", "content-length": "121", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
8062024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
8072024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
8082024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
8092024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
8102024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
8112024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
8122024-04-03T10:16:54.492ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
8132024-04-03T10:16:54.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8142024-04-03T10:16:54.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8152024-04-03T10:16:54.493ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8162024-04-03T10:16:54.494ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7cf8bcdb-ded6-400a-84f4-63c04103f7f7", "content-length": "2", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
8172024-04-03T10:16:54.494ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8182024-04-03T10:16:54.494ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9811a523-b128-4332-9ee9-51a1c55eb306", "content-length": "52", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
8192024-04-03T10:16:54.494ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
8202024-04-03T10:16:54.494ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8212024-04-03T10:16:54.542ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a3ab6641-2733-4a29-8e22-1f3729d6a436", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
8222024-04-03T10:16:54.542ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
8232024-04-03T10:16:54.542ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8242024-04-03T10:16:54.542ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4b34e171-f9ba-464c-bace-10c2796bb667", "content-length": "452", "date": "Wed, 03 Apr 2024 10:16:54 GMT"} })
8252024-04-03T10:16:54.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8262024-04-03T10:16:54.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8272024-04-03T10:16:54.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8282024-04-03T10:16:54.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8292024-04-03T10:16:54.548ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8302024-04-03T10:16:54.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8312024-04-03T10:16:54.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8322024-04-03T10:16:54.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8332024-04-03T10:16:54.554ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 435.230496ms iteration = 16
8342024-04-03T10:16:54.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 195 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 8f4e7402-6a39-4975-92ad-2b63f1fd61c8 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
8352024-04-03T10:17:04.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
8362024-04-03T10:17:04.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8372024-04-03T10:17:04.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8382024-04-03T10:17:04.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8392024-04-03T10:17:04.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8402024-04-03T10:17:04.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8412024-04-03T10:17:04.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8422024-04-03T10:17:04.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8432024-04-03T10:17:04.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8442024-04-03T10:17:04.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8452024-04-03T10:17:04.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
8462024-04-03T10:17:04.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8472024-04-03T10:17:04.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8482024-04-03T10:17:04.133ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
8492024-04-03T10:17:04.133ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.344352ms iteration = 45
8502024-04-03T10:17:04.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 189 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 3b9cba72-3e18-41cf-bd07-7c9d0ac693a0 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
8512024-04-03T10:17:14.119ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
8522024-04-03T10:17:14.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8532024-04-03T10:17:14.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8542024-04-03T10:17:14.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8552024-04-03T10:17:14.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8562024-04-03T10:17:14.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8572024-04-03T10:17:14.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8582024-04-03T10:17:14.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8592024-04-03T10:17:14.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8602024-04-03T10:17:14.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8612024-04-03T10:17:14.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
8622024-04-03T10:17:14.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8632024-04-03T10:17:14.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8642024-04-03T10:17:14.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
8652024-04-03T10:17:14.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.521283ms iteration = 46
8662024-04-03T10:17:14.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1165 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 3ad590c0-e286-479f-8059-988c0b929219 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
8672024-04-03T10:17:24.080ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
8682024-04-03T10:17:24.080ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
8692024-04-03T10:17:24.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
8702024-04-03T10:17:24.083ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1591, tv_nsec: 346799387 } })
8712024-04-03T10:17:24.083ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
8722024-04-03T10:17:24.118ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
8732024-04-03T10:17:24.118ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8742024-04-03T10:17:24.118ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "de5d5199-e87e-43a7-b02f-524b5aecbf29", "content-length": "26", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
8752024-04-03T10:17:24.118ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8762024-04-03T10:17:24.119ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8772024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
8782024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
8792024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
8802024-04-03T10:17:24.120ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
8812024-04-03T10:17:24.120ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
8822024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.56\u{b5}s iteration = 17
8832024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 17 reason = Timeout
8842024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8852024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8862024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8872024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
8882024-04-03T10:17:24.120ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
8892024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
8902024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8912024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8922024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8932024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8942024-04-03T10:17:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8952024-04-03T10:17:24.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8962024-04-03T10:17:24.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8972024-04-03T10:17:24.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8982024-04-03T10:17:24.123ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
8992024-04-03T10:17:24.123ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.505695ms iteration = 17
9002024-04-03T10:17:24.123ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1591, tv_nsec: 387196874 } })
9012024-04-03T10:17:24.123ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
9022024-04-03T10:17:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
9032024-04-03T10:17:24.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9042024-04-03T10:17:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9052024-04-03T10:17:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9062024-04-03T10:17:24.125ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9072024-04-03T10:17:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9082024-04-03T10:17:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9092024-04-03T10:17:24.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9102024-04-03T10:17:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9112024-04-03T10:17:24.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9122024-04-03T10:17:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9132024-04-03T10:17:24.127ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1591, tv_nsec: 390754939 } })
9142024-04-03T10:17:24.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9152024-04-03T10:17:24.127ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
9162024-04-03T10:17:24.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9172024-04-03T10:17:24.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9182024-04-03T10:17:24.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
9192024-04-03T10:17:24.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9202024-04-03T10:17:24.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9212024-04-03T10:17:24.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
9222024-04-03T10:17:24.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.932353ms iteration = 47
9232024-04-03T10:17:24.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9242024-04-03T10:17:24.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9252024-04-03T10:17:24.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9262024-04-03T10:17:24.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9272024-04-03T10:17:24.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9282024-04-03T10:17:24.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
9292024-04-03T10:17:24.146ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9302024-04-03T10:17:24.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9312024-04-03T10:17:24.147ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 27.337249ms iteration = 17
9322024-04-03T10:17:24.196ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 115.919736ms iteration = 17
9332024-04-03T10:17:24.199ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
9342024-04-03T10:17:24.199ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9352024-04-03T10:17:24.199ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "92248753-c914-4114-94c8-4a84b0d02b07", "content-length": "26", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
9362024-04-03T10:17:24.199ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9372024-04-03T10:17:24.200ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
9382024-04-03T10:17:24.200ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9392024-04-03T10:17:24.201ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6accdb89-2ae1-4fdb-a3da-02dfbab329aa", "content-length": "26", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
9402024-04-03T10:17:24.201ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9412024-04-03T10:17:24.201ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9422024-04-03T10:17:24.202ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
9432024-04-03T10:17:24.264ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9442024-04-03T10:17:24.264ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ccbf2a10-fc7c-401d-b913-8355c6f3c4f8", "content-length": "2", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
9452024-04-03T10:17:24.266ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9462024-04-03T10:17:24.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9472024-04-03T10:17:24.267ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 147.092536ms iteration = 17
9482024-04-03T10:17:24.325ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9492024-04-03T10:17:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9502024-04-03T10:17:24.326ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
9512024-04-03T10:17:24.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9522024-04-03T10:17:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9532024-04-03T10:17:24.332ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
9542024-04-03T10:17:24.333ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8bbc5462-cb12-4267-8699-4606bd066e9e", "content-length": "121", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
9552024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
9562024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
9572024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
9582024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
9592024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
9602024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
9612024-04-03T10:17:24.333ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
9622024-04-03T10:17:24.333ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9632024-04-03T10:17:24.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9642024-04-03T10:17:24.335ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
9652024-04-03T10:17:24.335ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f679bd1a-9c79-4222-b473-7e86e5c65365", "content-length": "2", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
9662024-04-03T10:17:24.335ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
9672024-04-03T10:17:24.336ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ea2f7236-0a92-480e-b12b-450146cd469c", "content-length": "52", "date": "Wed, 03 Apr 2024 10:17:24 GMT"} })
9682024-04-03T10:17:24.336ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
9692024-04-03T10:17:24.336ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
9702024-04-03T10:17:24.384ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3ffddb2f-2826-4a5b-aa4e-75e9f089cbd6", "date": "Wed, 03 Apr 2024 10:17:23 GMT"} })
9712024-04-03T10:17:24.384ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
9722024-04-03T10:17:24.384ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
9732024-04-03T10:17:24.428ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0bb24dae-59d6-4f32-b0d5-6cefa7e0132b", "content-length": "452", "date": "Wed, 03 Apr 2024 10:17:23 GMT"} })
9742024-04-03T10:17:24.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9752024-04-03T10:17:24.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9762024-04-03T10:17:24.430ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9772024-04-03T10:17:24.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9782024-04-03T10:17:24.433ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9792024-04-03T10:17:24.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9802024-04-03T10:17:24.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9812024-04-03T10:17:24.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9822024-04-03T10:17:24.440ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 319.532224ms iteration = 17
9832024-04-03T10:17:24.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 199 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 75e055f2-bd0a-4f4a-a343-32a425523e56 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
9842024-04-03T10:17:34.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
9852024-04-03T10:17:34.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9862024-04-03T10:17:34.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9872024-04-03T10:17:34.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9882024-04-03T10:17:34.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9892024-04-03T10:17:34.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9902024-04-03T10:17:34.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9912024-04-03T10:17:34.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9922024-04-03T10:17:34.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9932024-04-03T10:17:34.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9942024-04-03T10:17:34.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
9952024-04-03T10:17:34.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9962024-04-03T10:17:34.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9972024-04-03T10:17:34.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
9982024-04-03T10:17:34.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.033714ms iteration = 48
9992024-04-03T10:17:34.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 284 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 2a679e66-7c05-4c3f-acd0-4da9e0675d2d response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
10002024-04-03T10:17:44.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
10012024-04-03T10:17:44.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10022024-04-03T10:17:44.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10032024-04-03T10:17:44.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10042024-04-03T10:17:44.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10052024-04-03T10:17:44.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10062024-04-03T10:17:44.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10072024-04-03T10:17:44.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10082024-04-03T10:17:44.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10092024-04-03T10:17:44.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10102024-04-03T10:17:44.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
10112024-04-03T10:17:44.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10122024-04-03T10:17:44.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10132024-04-03T10:17:44.134ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
10142024-04-03T10:17:44.134ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.771731ms iteration = 49
10152024-04-03T10:17:44.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 296 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 6f6c17e4-ec39-4a7f-a0fa-b3e021edd332 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
10162024-04-03T10:17:54.079ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
10172024-04-03T10:17:54.079ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
10182024-04-03T10:17:54.079ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10192024-04-03T10:17:54.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10202024-04-03T10:17:54.079ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
10212024-04-03T10:17:54.079ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10222024-04-03T10:17:54.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10232024-04-03T10:17:54.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10242024-04-03T10:17:54.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10252024-04-03T10:17:54.081ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10262024-04-03T10:17:54.082ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1621, tv_nsec: 345652918 } })
10272024-04-03T10:17:54.082ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
10282024-04-03T10:17:54.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10292024-04-03T10:17:54.085ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10302024-04-03T10:17:54.085ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10312024-04-03T10:17:54.085ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10322024-04-03T10:17:54.085ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10332024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10342024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
10352024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
10362024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
10372024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
10382024-04-03T10:17:54.086ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
10392024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
10402024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 13.75\u{b5}s iteration = 10
10412024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
10422024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10432024-04-03T10:17:54.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10442024-04-03T10:17:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10452024-04-03T10:17:54.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10462024-04-03T10:17:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10472024-04-03T10:17:54.087ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e04e52c-490d-4b79-9cc6-e5adae4e5501.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("586c70fe-6c95-438f-be65-3daafd7be41b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1621, tv_nsec: 351231429 } })
10482024-04-03T10:17:54.088ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2fbdcbcd-ff1b-4375-856b-f7f3530b0e63.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1621, tv_nsec: 351232459 } })
10492024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
10502024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
10512024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
10522024-04-03T10:17:54.088ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27
10532024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.93\u{b5}s iteration = 11
10542024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10552024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10562024-04-03T10:17:54.088ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10572024-04-03T10:17:54.089ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10582024-04-03T10:17:54.090ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
10592024-04-03T10:17:54.090ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.106115ms iteration = 10
10602024-04-03T10:17:54.090ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
10612024-04-03T10:17:54.090ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.142105ms iteration = 10
10622024-04-03T10:17:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10632024-04-03T10:17:54.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10642024-04-03T10:17:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10652024-04-03T10:17:54.092ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10662024-04-03T10:17:54.092ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10672024-04-03T10:17:54.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10682024-04-03T10:17:54.093ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10692024-04-03T10:17:54.093ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10702024-04-03T10:17:54.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10712024-04-03T10:17:54.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10722024-04-03T10:17:54.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10732024-04-03T10:17:54.094ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10742024-04-03T10:17:54.094ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10752024-04-03T10:17:54.094ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10762024-04-03T10:17:54.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10772024-04-03T10:17:54.094ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10782024-04-03T10:17:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10792024-04-03T10:17:54.095ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10802024-04-03T10:17:54.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10812024-04-03T10:17:54.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10822024-04-03T10:17:54.098ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10832024-04-03T10:17:54.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10842024-04-03T10:17:54.099ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10852024-04-03T10:17:54.099ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10862024-04-03T10:17:54.100ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
10872024-04-03T10:17:54.101ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10882024-04-03T10:17:54.101ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-04-03 10:10:24.516388 UTC dns_group = internal generation = 1
10892024-04-03T10:17:54.101ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 22.003642ms iteration = 10
10902024-04-03T10:17:54.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10912024-04-03T10:17:54.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10922024-04-03T10:17:54.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10932024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10942024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10952024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10962024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
10972024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 0 version = 2
10982024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10992024-04-03T10:17:54.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11002024-04-03T10:17:54.105ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11012024-04-03T10:17:54.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11022024-04-03T10:17:54.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11032024-04-03T10:17:54.107ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11042024-04-03T10:17:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11052024-04-03T10:17:54.109ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11062024-04-03T10:17:54.109ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11072024-04-03T10:17:54.109ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11082024-04-03T10:17:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11092024-04-03T10:17:54.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11102024-04-03T10:17:54.111ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 1 version = 2
11112024-04-03T10:17:54.111ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11122024-04-03T10:17:54.111ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-04-03 10:10:25.138074 UTC dns_group = external generation = 2
11132024-04-03T10:17:54.111ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.487247ms iteration = 10
11142024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
11152024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
11162024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11172024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11182024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
11192024-04-03T10:17:54.120ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
11202024-04-03T10:17:54.120ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
11212024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
11222024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.621\u{b5}s iteration = 18
11232024-04-03T10:17:54.120ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
11242024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
11252024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 18 reason = Timeout
11262024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11272024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11282024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-04-03T10:09:54.267934Z, time_modified: 2024-04-03T10:09:54.267934Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }) }
11292024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 32.711498ms iteration = 10
11302024-04-03T10:17:54.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11312024-04-03T10:17:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11322024-04-03T10:17:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11332024-04-03T10:17:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11342024-04-03T10:17:54.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11352024-04-03T10:17:54.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11362024-04-03T10:17:54.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11372024-04-03T10:17:54.123ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1621, tv_nsec: 386432192 } })
11382024-04-03T10:17:54.123ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
11392024-04-03T10:17:54.123ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.751491ms iteration = 18
11402024-04-03T10:17:54.124ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
11412024-04-03T10:17:54.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11422024-04-03T10:17:54.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11432024-04-03T10:17:54.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11442024-04-03T10:17:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11452024-04-03T10:17:54.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11462024-04-03T10:17:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11472024-04-03T10:17:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11482024-04-03T10:17:54.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11492024-04-03T10:17:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11502024-04-03T10:17:54.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11512024-04-03T10:17:54.127ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1621, tv_nsec: 390664619 } })
11522024-04-03T10:17:54.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11532024-04-03T10:17:54.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11542024-04-03T10:17:54.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11552024-04-03T10:17:54.128ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
11562024-04-03T10:17:54.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11572024-04-03T10:17:54.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
11582024-04-03T10:17:54.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11592024-04-03T10:17:54.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11602024-04-03T10:17:54.136ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
11612024-04-03T10:17:54.136ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.451312ms iteration = 50
11622024-04-03T10:17:54.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11632024-04-03T10:17:54.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11642024-04-03T10:17:54.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11652024-04-03T10:17:54.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11662024-04-03T10:17:54.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11672024-04-03T10:17:54.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
11682024-04-03T10:17:54.144ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11692024-04-03T10:17:54.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11702024-04-03T10:17:54.145ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 24.693981ms iteration = 18
11712024-04-03T10:17:54.208ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
11722024-04-03T10:17:54.209ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
11732024-04-03T10:17:54.209ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11742024-04-03T10:17:54.209ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "15752269-d6d9-4762-a68a-3e42eaf3f372", "content-length": "26", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
11752024-04-03T10:17:54.209ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11762024-04-03T10:17:54.210ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11772024-04-03T10:17:54.213ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
11782024-04-03T10:17:54.272ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
11792024-04-03T10:17:54.272ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11802024-04-03T10:17:54.272ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "52630808-90c9-4197-bdaf-bb27db60d1d3", "content-length": "26", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
11812024-04-03T10:17:54.272ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11822024-04-03T10:17:54.274ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11832024-04-03T10:17:54.280ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
11842024-04-03T10:17:54.280ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11852024-04-03T10:17:54.281ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "955fda33-ab8d-434a-a081-0026eee924ae", "content-length": "26", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
11862024-04-03T10:17:54.281ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11872024-04-03T10:17:54.283ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
11882024-04-03T10:17:54.342ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 262.512622ms iteration = 18
11892024-04-03T10:17:54.344ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11902024-04-03T10:17:54.352ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11912024-04-03T10:17:54.353ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8ab9cec8-1e42-4925-8305-601e16fa2a55", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11922024-04-03T10:17:54.353ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11932024-04-03T10:17:54.355ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "745555f2-44d3-4d1d-b6c5-8715fb5e4feb", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11942024-04-03T10:17:54.355ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11952024-04-03T10:17:54.357ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11962024-04-03T10:17:54.357ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 269.370678ms iteration = 12
11972024-04-03T10:17:54.392ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11982024-04-03T10:17:54.392ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f1f71388-970c-4894-a9f2-a0660707901c", "content-length": "2", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
11992024-04-03T10:17:54.394ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12002024-04-03T10:17:54.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12012024-04-03T10:17:54.396ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 275.856496ms iteration = 18
12022024-04-03T10:17:54.407ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12032024-04-03T10:17:54.407ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0a77b98c-6234-4179-adcf-2921799decaf", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12042024-04-03T10:17:54.407ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12052024-04-03T10:17:54.409ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "af560163-4d62-48cf-8ee1-7557361208e3", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12062024-04-03T10:17:54.409ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12072024-04-03T10:17:54.410ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "079dd291-1147-4d22-9a28-d84e1d08507f", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12082024-04-03T10:17:54.410ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12092024-04-03T10:17:54.412ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
12102024-04-03T10:17:54.412ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 326.393669ms iteration = 12
12112024-04-03T10:17:54.454ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12122024-04-03T10:17:54.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12132024-04-03T10:17:54.455ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
12142024-04-03T10:17:54.455ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12152024-04-03T10:17:54.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12162024-04-03T10:17:54.462ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
12172024-04-03T10:17:54.462ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7581a7a1-1dfa-4b2c-8fa3-0ef17587ab88", "content-length": "121", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
12182024-04-03T10:17:54.462ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
12192024-04-03T10:17:54.462ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
12202024-04-03T10:17:54.463ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
12212024-04-03T10:17:54.463ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
12222024-04-03T10:17:54.463ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
12232024-04-03T10:17:54.463ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
12242024-04-03T10:17:54.463ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
12252024-04-03T10:17:54.463ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12262024-04-03T10:17:54.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12272024-04-03T10:17:54.465ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
12282024-04-03T10:17:54.465ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b07225f3-b206-49c5-9d4f-4ba308264fa2", "content-length": "2", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
12292024-04-03T10:17:54.465ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
12302024-04-03T10:17:54.466ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "96f6f00b-d112-482b-96ae-d1156ab41557", "content-length": "52", "date": "Wed, 03 Apr 2024 10:17:54 GMT"} })
12312024-04-03T10:17:54.466ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
12322024-04-03T10:17:54.466ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12332024-04-03T10:17:54.514ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "82cfb8f8-a489-43c3-858b-c32885416eec", "date": "Wed, 03 Apr 2024 10:17:53 GMT"} })
12342024-04-03T10:17:54.514ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
12352024-04-03T10:17:54.514ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12362024-04-03T10:17:54.515ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e0e007e2-dad4-4ce9-a186-0340ca595c6c", "content-length": "452", "date": "Wed, 03 Apr 2024 10:17:53 GMT"} })
12372024-04-03T10:17:54.515ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12382024-04-03T10:17:54.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12392024-04-03T10:17:54.517ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12402024-04-03T10:17:54.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12412024-04-03T10:17:54.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12422024-04-03T10:17:54.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12432024-04-03T10:17:54.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12442024-04-03T10:17:54.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12452024-04-03T10:17:54.528ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 407.553431ms iteration = 18
12462024-04-03T10:17:54.630ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 209 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 8abd64c0-ca85-4e2f-94ba-83cae9f3ab3c response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
12472024-04-03T10:18:04.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
12482024-04-03T10:18:04.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12492024-04-03T10:18:04.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12502024-04-03T10:18:04.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12512024-04-03T10:18:04.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12522024-04-03T10:18:04.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12532024-04-03T10:18:04.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12542024-04-03T10:18:04.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12552024-04-03T10:18:04.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12562024-04-03T10:18:04.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12572024-04-03T10:18:04.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
12582024-04-03T10:18:04.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12592024-04-03T10:18:04.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12602024-04-03T10:18:04.134ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
12612024-04-03T10:18:04.134ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.657269ms iteration = 51
12622024-04-03T10:18:04.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 285 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = c1cfa0b9-66fd-4b21-a573-270624d37c47 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
12632024-04-03T10:18:14.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
12642024-04-03T10:18:14.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12652024-04-03T10:18:14.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12662024-04-03T10:18:14.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12672024-04-03T10:18:14.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12682024-04-03T10:18:14.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12692024-04-03T10:18:14.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12702024-04-03T10:18:14.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12712024-04-03T10:18:14.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12722024-04-03T10:18:14.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12732024-04-03T10:18:14.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
12742024-04-03T10:18:14.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12752024-04-03T10:18:14.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12762024-04-03T10:18:14.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
12772024-04-03T10:18:14.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.194899ms iteration = 52
12782024-04-03T10:18:14.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 297 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 3c0d0d89-76f1-4621-b32e-4ebc124ee052 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
12792024-04-03T10:18:24.080ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
12802024-04-03T10:18:24.080ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12812024-04-03T10:18:24.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12822024-04-03T10:18:24.083ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1651, tv_nsec: 346458234 } })
12832024-04-03T10:18:24.083ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
12842024-04-03T10:18:24.113ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
12852024-04-03T10:18:24.113ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12862024-04-03T10:18:24.114ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c308109f-7e9c-40a5-8c54-6b7d61f6fc7e", "content-length": "26", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
12872024-04-03T10:18:24.114ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12882024-04-03T10:18:24.114ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12892024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
12902024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
12912024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
12922024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12932024-04-03T10:18:24.120ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
12942024-04-03T10:18:24.120ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
12952024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12962024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.88\u{b5}s iteration = 19
12972024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 19 reason = Timeout
12982024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
12992024-04-03T10:18:24.120ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
13002024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13012024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
13022024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13032024-04-03T10:18:24.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13042024-04-03T10:18:24.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13052024-04-03T10:18:24.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13062024-04-03T10:18:24.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13072024-04-03T10:18:24.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13082024-04-03T10:18:24.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13092024-04-03T10:18:24.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13102024-04-03T10:18:24.123ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1651, tv_nsec: 385962046 } })
13112024-04-03T10:18:24.123ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
13122024-04-03T10:18:24.123ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
13132024-04-03T10:18:24.123ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.73324ms iteration = 19
13142024-04-03T10:18:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13152024-04-03T10:18:24.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13162024-04-03T10:18:24.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13172024-04-03T10:18:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13182024-04-03T10:18:24.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13192024-04-03T10:18:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13202024-04-03T10:18:24.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13212024-04-03T10:18:24.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13222024-04-03T10:18:24.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13232024-04-03T10:18:24.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13242024-04-03T10:18:24.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13252024-04-03T10:18:24.128ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1651, tv_nsec: 391232432 } })
13262024-04-03T10:18:24.128ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
13272024-04-03T10:18:24.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13282024-04-03T10:18:24.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13292024-04-03T10:18:24.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13302024-04-03T10:18:24.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
13312024-04-03T10:18:24.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13322024-04-03T10:18:24.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13332024-04-03T10:18:24.136ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
13342024-04-03T10:18:24.136ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.136974ms iteration = 53
13352024-04-03T10:18:24.138ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13362024-04-03T10:18:24.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13372024-04-03T10:18:24.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13382024-04-03T10:18:24.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13392024-04-03T10:18:24.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13402024-04-03T10:18:24.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
13412024-04-03T10:18:24.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13422024-04-03T10:18:24.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13432024-04-03T10:18:24.144ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.834215ms iteration = 19
13442024-04-03T10:18:24.185ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 104.830234ms iteration = 19
13452024-04-03T10:18:24.202ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
13462024-04-03T10:18:24.202ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13472024-04-03T10:18:24.202ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ea00691c-77ee-4aad-ac1a-c551f09ce270", "content-length": "26", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13482024-04-03T10:18:24.202ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13492024-04-03T10:18:24.204ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13502024-04-03T10:18:24.206ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
13512024-04-03T10:18:24.206ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13522024-04-03T10:18:24.206ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "23bccbb5-d5a6-43e0-9969-441d60e34f78", "content-length": "26", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13532024-04-03T10:18:24.206ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13542024-04-03T10:18:24.208ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
13552024-04-03T10:18:24.263ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
13562024-04-03T10:18:24.263ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "517bfe37-5598-48a6-a151-a6e3f28707ad", "content-length": "2", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13572024-04-03T10:18:24.263ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13582024-04-03T10:18:24.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13592024-04-03T10:18:24.265ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 144.719512ms iteration = 19
13602024-04-03T10:18:24.327ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13612024-04-03T10:18:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13622024-04-03T10:18:24.328ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
13632024-04-03T10:18:24.328ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13642024-04-03T10:18:24.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13652024-04-03T10:18:24.335ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
13662024-04-03T10:18:24.335ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f805c605-d833-4fa2-b4c7-6d7607ab5897", "content-length": "121", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13672024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
13682024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
13692024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
13702024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
13712024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
13722024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
13732024-04-03T10:18:24.335ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
13742024-04-03T10:18:24.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13752024-04-03T10:18:24.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13762024-04-03T10:18:24.337ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
13772024-04-03T10:18:24.337ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "72cf264d-4cc3-4d63-8857-7891744d1c84", "content-length": "2", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13782024-04-03T10:18:24.337ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
13792024-04-03T10:18:24.337ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "44e7f578-512a-44bd-8d30-0714c5bf100d", "content-length": "52", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13802024-04-03T10:18:24.337ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
13812024-04-03T10:18:24.337ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
13822024-04-03T10:18:24.384ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6acda432-8b02-42ba-8a06-d30426c5a019", "date": "Wed, 03 Apr 2024 10:18:23 GMT"} })
13832024-04-03T10:18:24.384ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
13842024-04-03T10:18:24.384ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13852024-04-03T10:18:24.388ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "26673b6d-b138-42ba-9dff-1c08a22f8403", "content-length": "452", "date": "Wed, 03 Apr 2024 10:18:24 GMT"} })
13862024-04-03T10:18:24.388ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13872024-04-03T10:18:24.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13882024-04-03T10:18:24.391ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13892024-04-03T10:18:24.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13902024-04-03T10:18:24.393ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13912024-04-03T10:18:24.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13922024-04-03T10:18:24.395ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13932024-04-03T10:18:24.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13942024-04-03T10:18:24.401ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 280.580592ms iteration = 19
13952024-04-03T10:18:24.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 1269 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 57969566-3c84-483d-ba51-7278b2df5804 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
13962024-04-03T10:18:34.121ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
13972024-04-03T10:18:34.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13982024-04-03T10:18:34.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13992024-04-03T10:18:34.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14002024-04-03T10:18:34.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14012024-04-03T10:18:34.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14022024-04-03T10:18:34.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14032024-04-03T10:18:34.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14042024-04-03T10:18:34.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14052024-04-03T10:18:34.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14062024-04-03T10:18:34.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
14072024-04-03T10:18:34.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14082024-04-03T10:18:34.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14092024-04-03T10:18:34.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
14102024-04-03T10:18:34.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 13.877063ms iteration = 54
14112024-04-03T10:18:34.631ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 201 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 5ac12a87-85ed-48cd-b442-797171449f17 response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
14122024-04-03T10:18:44.121ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
14132024-04-03T10:18:44.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14142024-04-03T10:18:44.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14152024-04-03T10:18:44.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14162024-04-03T10:18:44.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14172024-04-03T10:18:44.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14182024-04-03T10:18:44.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14192024-04-03T10:18:44.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14202024-04-03T10:18:44.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14212024-04-03T10:18:44.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14222024-04-03T10:18:44.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
14232024-04-03T10:18:44.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14242024-04-03T10:18:44.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14252024-04-03T10:18:44.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
14262024-04-03T10:18:44.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.134928ms iteration = 55
14272024-04-03T10:18:44.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 809 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = 42087fe9-501e-4eec-a067-8312b2b383ce response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004
14282024-04-03T10:18:54.079ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
14292024-04-03T10:18:54.079ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
14302024-04-03T10:18:54.079ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14312024-04-03T10:18:54.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14322024-04-03T10:18:54.080ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14332024-04-03T10:18:54.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14342024-04-03T10:18:54.081ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
14352024-04-03T10:18:54.081ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14362024-04-03T10:18:54.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14372024-04-03T10:18:54.082ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14382024-04-03T10:18:54.083ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1681, tv_nsec: 345658881 } })
14392024-04-03T10:18:54.083ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:974
14402024-04-03T10:18:54.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14412024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
14422024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
14432024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
14442024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14452024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14462024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
14472024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
14482024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
14492024-04-03T10:18:54.086ZWARNnexus (BackgroundTasks): Metric producer GC: statically disabled pending omicron#5284 background_task = metrics_producer_gc file = nexus/src/app/background/metrics_producer_gc.rs:43
14502024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
14512024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 14.661\u{b5}s iteration = 11
14522024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
14532024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
14542024-04-03T10:18:54.086ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14552024-04-03T10:18:54.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14562024-04-03T10:18:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14572024-04-03T10:18:54.087ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14582024-04-03T10:18:54.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14592024-04-03T10:18:54.088ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0e04e52c-490d-4b79-9cc6-e5adae4e5501.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("586c70fe-6c95-438f-be65-3daafd7be41b.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("c60e57f5-2712-45fb-af0b-bc0ff46e083c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1681, tv_nsec: 350696143 } })
14602024-04-03T10:18:54.088ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2fbdcbcd-ff1b-4375-856b-f7f3530b0e63.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("ad48337a-add3-4214-9d0f-f8db41e43622.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1681, tv_nsec: 350770254 } })
14612024-04-03T10:18:54.088ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
14622024-04-03T10:18:54.088ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
14632024-04-03T10:18:54.088ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/blueprint_execution.rs:65 reason = blueprint disabled target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27
14642024-04-03T10:18:54.088ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
14652024-04-03T10:18:54.088ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 10.141\u{b5}s iteration = 12
14662024-04-03T10:18:54.088ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14672024-04-03T10:18:54.089ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14682024-04-03T10:18:54.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14692024-04-03T10:18:54.089ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
14702024-04-03T10:18:54.089ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.003854ms iteration = 11
14712024-04-03T10:18:54.090ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14722024-04-03T10:18:54.091ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
14732024-04-03T10:18:54.091ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.276498ms iteration = 11
14742024-04-03T10:18:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14752024-04-03T10:18:54.091ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14762024-04-03T10:18:54.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14772024-04-03T10:18:54.093ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
14782024-04-03T10:18:54.093ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14792024-04-03T10:18:54.093ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14802024-04-03T10:18:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14812024-04-03T10:18:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14822024-04-03T10:18:54.094ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14832024-04-03T10:18:54.094ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14842024-04-03T10:18:54.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14852024-04-03T10:18:54.095ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14862024-04-03T10:18:54.095ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
14872024-04-03T10:18:54.095ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
14882024-04-03T10:18:54.095ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
14892024-04-03T10:18:54.095ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14902024-04-03T10:18:54.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14912024-04-03T10:18:54.096ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14922024-04-03T10:18:54.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14932024-04-03T10:18:54.097ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14942024-04-03T10:18:54.097ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14952024-04-03T10:18:54.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14962024-04-03T10:18:54.100ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14972024-04-03T10:18:54.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14982024-04-03T10:18:54.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
14992024-04-03T10:18:54.101ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15002024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15012024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15022024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15032024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 3cbd5c17-7023-4414-98e4-0c7a3ad83579 dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
15042024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
15052024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-04-03 10:10:24.516388 UTC dns_group = internal generation = 1
15062024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 23.01412ms iteration = 11
15072024-04-03T10:18:54.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15082024-04-03T10:18:54.103ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15092024-04-03T10:18:54.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15102024-04-03T10:18:54.104ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
15112024-04-03T10:18:54.104ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 0 version = 2
15122024-04-03T10:18:54.104ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15132024-04-03T10:18:54.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15142024-04-03T10:18:54.104ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15152024-04-03T10:18:54.105ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15162024-04-03T10:18:54.108ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
15172024-04-03T10:18:54.108ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15182024-04-03T10:18:54.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15192024-04-03T10:18:54.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
15202024-04-03T10:18:54.110ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15212024-04-03T10:18:54.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
15222024-04-03T10:18:54.111ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = d621cc8a-9f55-4094-86b5-5a73f96daa18 dns_zone_name = oxide.test found_so_far = 1 version = 2
15232024-04-03T10:18:54.111ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
15242024-04-03T10:18:54.111ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-04-03 10:10:25.138074 UTC dns_group = external generation = 2
15252024-04-03T10:18:54.111ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 24.948735ms iteration = 11
15262024-04-03T10:18:54.114ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-04-03T10:09:54.267934Z, time_modified: 2024-04-03T10:09:54.267934Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, db_silo: Silo { identity: SiloIdentity { id: e7be7de3-b664-4c93-bbb6-1cdb9ebff339, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-04-03T10:10:25.114689Z, time_modified: 2024-04-03T10:10:25.114689Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "d0ab64c58c956a6cfbef0f887c016075378eee1823ba8d9abd71fe6a876797fe" }] }) }
15272024-04-03T10:18:54.114ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 25.265932ms iteration = 11
15282024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
15292024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
15302024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
15312024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15322024-04-03T10:18:54.121ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:44
15332024-04-03T10:18:54.121ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/region_replacement.rs:48
15342024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 4.051\u{b5}s iteration = 20
15352024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15362024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 20 reason = Timeout
15372024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15382024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
15392024-04-03T10:18:54.121ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:46
15402024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
15412024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15422024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
15432024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15442024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15452024-04-03T10:18:54.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15462024-04-03T10:18:54.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15472024-04-03T10:18:54.122ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15482024-04-03T10:18:54.122ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1681, tv_nsec: 385405456 } })
15492024-04-03T10:18:54.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15502024-04-03T10:18:54.123ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:974
15512024-04-03T10:18:54.124ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/phantom_disks.rs:92
15522024-04-03T10:18:54.124ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.628286ms iteration = 20
15532024-04-03T10:18:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15542024-04-03T10:18:54.126ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15552024-04-03T10:18:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15562024-04-03T10:18:54.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15572024-04-03T10:18:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15582024-04-03T10:18:54.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15592024-04-03T10:18:54.127ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15602024-04-03T10:18:54.127ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15612024-04-03T10:18:54.127ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15622024-04-03T10:18:54.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15632024-04-03T10:18:54.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15642024-04-03T10:18:54.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15652024-04-03T10:18:54.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15662024-04-03T10:18:54.128ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-aa5c4df3-cd22-4b16-be8c-4ae0b07c237e.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1681, tv_nsec: 391317924 } })
15672024-04-03T10:18:54.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15682024-04-03T10:18:54.129ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:974 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
15692024-04-03T10:18:54.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: f14ac986-2d5f-4dab-85d2-58f0fca91a27, lookup_type: ById(f14ac986-2d5f-4dab-85d2-58f0fca91a27) } result = Ok(())
15702024-04-03T10:18:54.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15712024-04-03T10:18:54.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15722024-04-03T10:18:54.135ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 original_time_created = 2024-04-03 10:09:40.753058 UTC target_id = f14ac986-2d5f-4dab-85d2-58f0fca91a27 time_created = 2024-04-03 10:09:40.753058 UTC
15732024-04-03T10:18:54.135ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.495125ms iteration = 56
15742024-04-03T10:18:54.139ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15752024-04-03T10:18:54.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15762024-04-03T10:18:54.140ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15772024-04-03T10:18:54.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15782024-04-03T10:18:54.141ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15792024-04-03T10:18:54.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: aa5c4df3-cd22-4b16-be8c-4ae0b07c237e, lookup_type: ById(aa5c4df3-cd22-4b16-be8c-4ae0b07c237e) } result = Ok(())
15802024-04-03T10:18:54.143ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15812024-04-03T10:18:54.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15822024-04-03T10:18:54.145ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 24.391236ms iteration = 20
15832024-04-03T10:18:54.192ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
15842024-04-03T10:18:54.192ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15852024-04-03T10:18:54.194ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f6ee63ef-8849-4fe7-88e2-090890eab8bf", "content-length": "26", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
15862024-04-03T10:18:54.194ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15872024-04-03T10:18:54.196ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15882024-04-03T10:18:54.209ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
15892024-04-03T10:18:54.209ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
15902024-04-03T10:18:54.268ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:982 zone_address = fd00:1122:3344:101::2
15912024-04-03T10:18:54.268ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15922024-04-03T10:18:54.268ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4c93524e-a75f-4209-9d2d-90ea10fe613d", "content-length": "26", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
15932024-04-03T10:18:54.268ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:987 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15942024-04-03T10:18:54.269ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
15952024-04-03T10:18:54.332ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:982 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 zone_address = fd00:1122:3344:101::2
15962024-04-03T10:18:54.332ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15972024-04-03T10:18:54.334ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9b0a2b4d-ef24-4ca0-8c0b-0940b9b5ab6a", "content-length": "26", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
15982024-04-03T10:18:54.334ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:987 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15992024-04-03T10:18:54.335ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1019 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
16002024-04-03T10:18:54.352ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
16012024-04-03T10:18:54.352ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5d40540a-4ab3-49ed-8dd3-6e874607510e", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16022024-04-03T10:18:54.352ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16032024-04-03T10:18:54.354ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4175d3a8-694b-4477-96ab-56b9fa5fdeb4", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16042024-04-03T10:18:54.354ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16052024-04-03T10:18:54.355ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
16062024-04-03T10:18:54.355ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 266.75311ms iteration = 13
16072024-04-03T10:18:54.364ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
16082024-04-03T10:18:54.388ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
16092024-04-03T10:18:54.388ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "58cabd52-fc48-45d2-9bbf-bae2553c58e0", "content-length": "2", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
16102024-04-03T10:18:54.391ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16112024-04-03T10:18:54.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16122024-04-03T10:18:54.393ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 272.426654ms iteration = 20
16132024-04-03T10:18:54.394ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 313.329631ms iteration = 20
16142024-04-03T10:18:54.419ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
16152024-04-03T10:18:54.419ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "048890f1-65a6-4e7a-85b1-5e28d703b1a3", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16162024-04-03T10:18:54.419ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16172024-04-03T10:18:54.420ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "89050f76-98ca-44ea-9fe9-8f0c5b4d89f0", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16182024-04-03T10:18:54.420ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16192024-04-03T10:18:54.421ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "e10e463f-fd6b-437f-b2c3-db03bbbe134b", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16202024-04-03T10:18:54.421ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16212024-04-03T10:18:54.423ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
16222024-04-03T10:18:54.423ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 336.895521ms iteration = 13
16232024-04-03T10:18:54.471ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16242024-04-03T10:18:54.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16252024-04-03T10:18:54.472ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:141 port = SwitchPort { id: 50d22ede-a8e0-4aaa-90d3-27dd03be0905, rack_id: b85ae885-2925-429b-842f-b57ce9e9b6a0, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(c930a900-929e-4bbb-9989-cd57878b12e2) } rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
16262024-04-03T10:18:54.472ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16272024-04-03T10:18:54.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16282024-04-03T10:18:54.479ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
16292024-04-03T10:18:54.479ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b1f7b92d-aeba-44af-b59e-46f8f670cadf", "content-length": "121", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
16302024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1418 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
16312024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:1445 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = c930a900-929e-4bbb-9989-cd57878b12e2
16322024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:364 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
16332024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:368 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 })}}
16342024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:377 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
16352024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:383 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 routes = {}
16362024-04-03T10:18:54.479ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:403 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0
16372024-04-03T10:18:54.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16382024-04-03T10:18:54.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16392024-04-03T10:18:54.481ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
16402024-04-03T10:18:54.481ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d30543cc-fcca-4fd7-93f6-f3d69ea325bd", "content-length": "2", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
16412024-04-03T10:18:54.481ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
16422024-04-03T10:18:54.481ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b06fbb53-6c0b-4c66-b1ca-94708c2283ac", "content-length": "52", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
16432024-04-03T10:18:54.481ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [V4(Ipv4Network { addr: 10.151.1.174, prefix: 24 })], port: "qsfp0" }] file = nexus/src/app/background/sync_switch_configuration.rs:452 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch_location = Switch0
16442024-04-03T10:18:54.481ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
16452024-04-03T10:18:54.529ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "01ff77fb-a78c-448f-b534-279ccbe16913", "date": "Wed, 03 Apr 2024 10:18:53 GMT"} })
16462024-04-03T10:18:54.529ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/sync_switch_configuration.rs:617 rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 switch = Switch0
16472024-04-03T10:18:54.529ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
16482024-04-03T10:18:54.529ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b85ae885-2925-429b-842f-b57ce9e9b6a0 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7b9f4373-d041-4b87-acde-6b6abc88bd42", "content-length": "452", "date": "Wed, 03 Apr 2024 10:18:54 GMT"} })
16492024-04-03T10:18:54.529ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16502024-04-03T10:18:54.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16512024-04-03T10:18:54.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16522024-04-03T10:18:54.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16532024-04-03T10:18:54.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16542024-04-03T10:18:54.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16552024-04-03T10:18:54.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16562024-04-03T10:18:54.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16572024-04-03T10:18:54.543ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 422.209449ms iteration = 20
16582024-04-03T10:18:54.632ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/29ae98d/dropshot/src/server.rs:849 latency_us = 923 local_addr = [fd00:1122:3344:101::c]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:63670 req_id = ba5c0564-fb94-46b8-b64e-436823c0ef8c response_code = 200 uri = /metrics/collect/e35963e6-a05d-4aaa-a3e2-d78509c59004