Max level shown:
12024-07-19T23:00:06.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
22024-07-19T23:00:06.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32024-07-19T23:00:06.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42024-07-19T23:00:06.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
52024-07-19T23:00:06.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
62024-07-19T23:00:06.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72024-07-19T23:00:06.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82024-07-19T23:00:06.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92024-07-19T23:00:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
102024-07-19T23:00:06.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112024-07-19T23:00:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122024-07-19T23:00:06.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132024-07-19T23:00:06.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
142024-07-19T23:00:06.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152024-07-19T23:00:06.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
162024-07-19T23:00:06.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172024-07-19T23:00:06.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
182024-07-19T23:00:06.501ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
192024-07-19T23:00:06.501ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.957978ms iteration = 15 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
202024-07-19T23:00:06.510ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
212024-07-19T23:00:06.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
222024-07-19T23:00:06.511ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
232024-07-19T23:00:06.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 28.109234ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
242024-07-19T23:00:07.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 395 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = d97c722a-c88d-425a-b339-283f5b011751 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
252024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 6 reason = Timeout
262024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
272024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
282024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
292024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
302024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
312024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
322024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
332024-07-19T23:00:16.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
342024-07-19T23:00:16.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
352024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
362024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
372024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
382024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
392024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
402024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
412024-07-19T23:00:16.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
422024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
432024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
442024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
452024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
462024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
472024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
482024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
492024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
502024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
512024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
522024-07-19T23:00:16.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
532024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
542024-07-19T23:00:16.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
552024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
562024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
572024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
582024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
592024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
602024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
612024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
622024-07-19T23:00:16.484ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
632024-07-19T23:00:16.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
642024-07-19T23:00:16.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
652024-07-19T23:00:16.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2238, tv_nsec: 682109895 } })
662024-07-19T23:00:16.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2238, tv_nsec: 682179226 } })
672024-07-19T23:00:16.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
682024-07-19T23:00:16.486ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
692024-07-19T23:00:16.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
702024-07-19T23:00:16.487ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
712024-07-19T23:00:16.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.254008ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
722024-07-19T23:00:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
732024-07-19T23:00:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
742024-07-19T23:00:16.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
752024-07-19T23:00:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
762024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
772024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
782024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
792024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
802024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
812024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
822024-07-19T23:00:16.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
832024-07-19T23:00:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
842024-07-19T23:00:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
852024-07-19T23:00:16.490ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
862024-07-19T23:00:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
872024-07-19T23:00:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
882024-07-19T23:00:16.491ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
892024-07-19T23:00:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
902024-07-19T23:00:16.494ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
912024-07-19T23:00:16.494ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
922024-07-19T23:00:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
932024-07-19T23:00:16.495ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
942024-07-19T23:00:16.495ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 11.886067ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
952024-07-19T23:00:16.496ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2238, tv_nsec: 692310281 } })
962024-07-19T23:00:16.496ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
972024-07-19T23:00:16.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
982024-07-19T23:00:16.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
992024-07-19T23:00:16.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1002024-07-19T23:00:16.499ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1012024-07-19T23:00:16.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1022024-07-19T23:00:16.499ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1032024-07-19T23:00:16.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1042024-07-19T23:00:16.500ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1052024-07-19T23:00:16.500ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1062024-07-19T23:00:16.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1072024-07-19T23:00:16.501ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1082024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1092024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1102024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
1112024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1122024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1132024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1142024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1152024-07-19T23:00:16.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1162024-07-19T23:00:16.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1172024-07-19T23:00:16.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1182024-07-19T23:00:16.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1192024-07-19T23:00:16.512ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
1202024-07-19T23:00:16.512ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.102771ms iteration = 16 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
1212024-07-19T23:00:16.513ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
1222024-07-19T23:00:16.513ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 29.761174ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
1232024-07-19T23:00:16.516ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1242024-07-19T23:00:16.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1252024-07-19T23:00:16.518ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1262024-07-19T23:00:16.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1272024-07-19T23:00:16.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1282024-07-19T23:00:16.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1292024-07-19T23:00:16.521ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1302024-07-19T23:00:16.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1312024-07-19T23:00:16.522ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1322024-07-19T23:00:16.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1332024-07-19T23:00:16.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1342024-07-19T23:00:16.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1352024-07-19T23:00:16.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1362024-07-19T23:00:16.525ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1372024-07-19T23:00:16.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1382024-07-19T23:00:16.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
1392024-07-19T23:00:16.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1402024-07-19T23:00:16.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1412024-07-19T23:00:16.528ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 44.687077ms iteration = 6 status = 0
1422024-07-19T23:00:16.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1432024-07-19T23:00:16.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1442024-07-19T23:00:16.541ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
1452024-07-19T23:00:16.541ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 57.96867ms iteration = 6 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
1462024-07-19T23:00:16.543ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
1472024-07-19T23:00:16.543ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 60.988124ms iteration = 7 status = {"physical_disks_added":0}
1482024-07-19T23:00:16.615ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
1492024-07-19T23:00:16.615ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1502024-07-19T23:00:16.615ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "87099ba8-7509-4d14-825f-4577cc852adf", "content-length": "26", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1512024-07-19T23:00:16.615ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1522024-07-19T23:00:16.618ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1532024-07-19T23:00:16.643ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
1542024-07-19T23:00:16.643ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1552024-07-19T23:00:16.644ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f886df76-bf21-42c4-ab0f-41a5e9345d11", "content-length": "26", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1562024-07-19T23:00:16.644ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1572024-07-19T23:00:16.645ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
1582024-07-19T23:00:16.698ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1592024-07-19T23:00:16.700ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5da650a4-60a6-4f05-b8ac-93921d444ff1", "content-length": "691", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1602024-07-19T23:00:16.700ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
1612024-07-19T23:00:16.700ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
1622024-07-19T23:00:16.700ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
1632024-07-19T23:00:16.701ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 217.68875ms iteration = 6 status = {}
1642024-07-19T23:00:16.720ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
1652024-07-19T23:00:16.720ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1662024-07-19T23:00:16.720ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7b80740b-dbaa-4f3f-9282-cd4695ddd34e", "content-length": "26", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1672024-07-19T23:00:16.721ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1682024-07-19T23:00:16.722ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1692024-07-19T23:00:16.743ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1702024-07-19T23:00:16.743ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d32fa4d3-905b-468c-8884-e41b8b5468fb", "content-length": "372", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1712024-07-19T23:00:16.743ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1722024-07-19T23:00:16.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1732024-07-19T23:00:16.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1742024-07-19T23:00:16.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1752024-07-19T23:00:16.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1762024-07-19T23:00:16.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1772024-07-19T23:00:16.750ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1782024-07-19T23:00:16.750ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "206f1a34-b037-4f43-ae76-c997852df702", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1792024-07-19T23:00:16.751ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 268.07747ms iteration = 6 status = {}
1802024-07-19T23:00:16.826ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1812024-07-19T23:00:16.826ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "aac0a872-2e07-4666-ad74-6eb138937ef7", "content-length": "1", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1822024-07-19T23:00:16.828ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1832024-07-19T23:00:16.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1842024-07-19T23:00:16.830ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 347.314548ms iteration = 6 status = null
1852024-07-19T23:00:16.850ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 367.593708ms iteration = 6 status = {}
1862024-07-19T23:00:16.909ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1872024-07-19T23:00:16.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1882024-07-19T23:00:16.911ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
1892024-07-19T23:00:16.911ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1902024-07-19T23:00:16.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1912024-07-19T23:00:16.919ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
1922024-07-19T23:00:16.919ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2e4c1f88-47da-41a9-b6cf-411baf6ddd73", "content-length": "121", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
1932024-07-19T23:00:16.919ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
1942024-07-19T23:00:16.919ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
1952024-07-19T23:00:16.920ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
1962024-07-19T23:00:16.920ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
1972024-07-19T23:00:16.920ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
1982024-07-19T23:00:16.920ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
1992024-07-19T23:00:16.920ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
2002024-07-19T23:00:16.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2012024-07-19T23:00:16.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2022024-07-19T23:00:16.922ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2032024-07-19T23:00:16.923ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "62d62034-d8b7-4181-828b-bb0463ed3608", "content-length": "2", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
2042024-07-19T23:00:16.923ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2052024-07-19T23:00:16.923ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8299790d-f784-46c1-8d3f-fbe49301b0e7", "content-length": "52", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
2062024-07-19T23:00:16.923ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
2072024-07-19T23:00:16.923ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2082024-07-19T23:00:17.007ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8615dbfe-e11e-425c-a816-a5e40f4b803e", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
2092024-07-19T23:00:17.007ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
2102024-07-19T23:00:17.007ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2112024-07-19T23:00:17.007ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fc80a0d5-b995-4763-979f-47227046d9e3", "content-length": "494", "date": "Fri, 19 Jul 2024 23:00:16 GMT"} })
2122024-07-19T23:00:17.007ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2132024-07-19T23:00:17.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2142024-07-19T23:00:17.011ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2152024-07-19T23:00:17.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2162024-07-19T23:00:17.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 901 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = d680b0dd-527f-48d0-b4a8-afefbbd56477 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
2172024-07-19T23:00:17.013ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2182024-07-19T23:00:17.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2192024-07-19T23:00:17.014ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2202024-07-19T23:00:17.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2212024-07-19T23:00:17.021ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 537.602585ms iteration = 6 status = {}
2222024-07-19T23:00:26.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
2232024-07-19T23:00:26.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2242024-07-19T23:00:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2252024-07-19T23:00:26.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
2262024-07-19T23:00:26.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
2272024-07-19T23:00:26.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2282024-07-19T23:00:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2292024-07-19T23:00:26.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2302024-07-19T23:00:26.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2312024-07-19T23:00:26.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2322024-07-19T23:00:26.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
2332024-07-19T23:00:26.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.28396ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2342024-07-19T23:00:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2352024-07-19T23:00:26.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2362024-07-19T23:00:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2372024-07-19T23:00:26.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2382024-07-19T23:00:26.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2392024-07-19T23:00:26.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2402024-07-19T23:00:26.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
2412024-07-19T23:00:26.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2422024-07-19T23:00:26.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2432024-07-19T23:00:26.497ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
2442024-07-19T23:00:26.497ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 14.616098ms iteration = 17 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
2452024-07-19T23:00:27.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 928 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 37609a6a-be74-4a47-9c69-3b825586adba response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
2462024-07-19T23:00:36.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
2472024-07-19T23:00:36.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2482024-07-19T23:00:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2492024-07-19T23:00:36.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
2502024-07-19T23:00:36.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
2512024-07-19T23:00:36.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2522024-07-19T23:00:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2532024-07-19T23:00:36.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2542024-07-19T23:00:36.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2552024-07-19T23:00:36.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2562024-07-19T23:00:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2572024-07-19T23:00:36.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2582024-07-19T23:00:36.488ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
2592024-07-19T23:00:36.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.564723ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2602024-07-19T23:00:36.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2612024-07-19T23:00:36.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2622024-07-19T23:00:36.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2632024-07-19T23:00:36.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2642024-07-19T23:00:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
2652024-07-19T23:00:36.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2662024-07-19T23:00:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2672024-07-19T23:00:36.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
2682024-07-19T23:00:36.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.155124ms iteration = 18 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
2692024-07-19T23:00:37.012ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 357 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 32cd1ecd-1b31-4c2c-a90c-0e8995f37803 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
2702024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
2712024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2722024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
2732024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2742024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2752024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
2762024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2772024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
2782024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2792024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
2802024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
2812024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
2822024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
2832024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2842024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
2852024-07-19T23:00:46.483ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-07-19 22:50:46.483346232 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2862024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
2872024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2882024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
2892024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2902024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
2912024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2922024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 7 reason = Timeout
2932024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2942024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
2952024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
2962024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
2972024-07-19T23:00:46.483ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
2982024-07-19T23:00:46.483ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:69 reason = blueprint disabled target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3
2992024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
3002024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
3012024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 17.29\u{b5}s iteration = 5 status = {"error":"blueprint disabled","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3"}
3022024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
3032024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
3042024-07-19T23:00:46.483ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
3052024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3062024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3072024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
3082024-07-19T23:00:46.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
3092024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3102024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3112024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3122024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
3132024-07-19T23:00:46.483ZINFOnexus (BackgroundTasks): lookup region port task started background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:56
3142024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3152024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
3162024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3172024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
3182024-07-19T23:00:46.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
3192024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3202024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
3212024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
3222024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3232024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
3242024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
3252024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3262024-07-19T23:00:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3272024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3282024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3292024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3302024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3312024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3322024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
3332024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3342024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3352024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3362024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3372024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3382024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
3392024-07-19T23:00:46.484ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d7897bb-e108-4529-b11c-7e55a9a521fc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4ec1dd64-fc59-46ad-8f12-dcafd769aba1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 2268, tv_nsec: 680179691 } })
3402024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3412024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3422024-07-19T23:00:46.484ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0f5ab007-5c53-4a1d-880a-afbba1f1b219.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 2268, tv_nsec: 680222372 } })
3432024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3442024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3452024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3462024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3472024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3482024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3492024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3502024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3512024-07-19T23:00:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3522024-07-19T23:00:46.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3532024-07-19T23:00:46.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3542024-07-19T23:00:46.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3552024-07-19T23:00:46.486ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3562024-07-19T23:00:46.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
3582024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.81707ms iteration = 4 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
3592024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
3602024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.940552ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3612024-07-19T23:00:46.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2268, tv_nsec: 683305998 } })
3622024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
3632024-07-19T23:00:46.487ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
3642024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 3.962852ms iteration = 4 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
3652024-07-19T23:00:46.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
3662024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3672024-07-19T23:00:46.487ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
3682024-07-19T23:00:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.465272ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
3692024-07-19T23:00:46.488ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3702024-07-19T23:00:46.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3712024-07-19T23:00:46.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3722024-07-19T23:00:46.489ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2268, tv_nsec: 685294925 } })
3732024-07-19T23:00:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3742024-07-19T23:00:46.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3752024-07-19T23:00:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3762024-07-19T23:00:46.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
3772024-07-19T23:00:46.490ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
3782024-07-19T23:00:46.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
3792024-07-19T23:00:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3802024-07-19T23:00:46.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3812024-07-19T23:00:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3822024-07-19T23:00:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3832024-07-19T23:00:46.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3842024-07-19T23:00:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3852024-07-19T23:00:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3862024-07-19T23:00:46.495ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3872024-07-19T23:00:46.496ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
3882024-07-19T23:00:46.496ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
3892024-07-19T23:00:46.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 13.279132ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3902024-07-19T23:00:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3912024-07-19T23:00:46.498ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3922024-07-19T23:00:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3932024-07-19T23:00:46.501ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3942024-07-19T23:00:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3952024-07-19T23:00:46.501ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
3962024-07-19T23:00:46.501ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
3972024-07-19T23:00:46.501ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
3982024-07-19T23:00:46.501ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3992024-07-19T23:00:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4002024-07-19T23:00:46.502ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
4012024-07-19T23:00:46.502ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4022024-07-19T23:00:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4032024-07-19T23:00:46.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4042024-07-19T23:00:46.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4052024-07-19T23:00:46.503ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4062024-07-19T23:00:46.503ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
4072024-07-19T23:00:46.503ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 19.923904ms iteration = 7 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
4082024-07-19T23:00:46.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4092024-07-19T23:00:46.504ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4102024-07-19T23:00:46.504ZINFOnexus (BackgroundTasks): lookup region port task done background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:150
4112024-07-19T23:00:46.504ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.82406ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
4122024-07-19T23:00:46.505ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-07-19 22:50:46.483346232 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4132024-07-19T23:00:46.505ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 21.88927ms iteration = 4 status = {"expiration":"2024-07-19T22:50:46.483346232Z","pruned":[]}
4142024-07-19T23:00:46.505ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4152024-07-19T23:00:46.505ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4162024-07-19T23:00:46.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4172024-07-19T23:00:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4182024-07-19T23:00:46.506ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4192024-07-19T23:00:46.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4202024-07-19T23:00:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4212024-07-19T23:00:46.506ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4222024-07-19T23:00:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
4232024-07-19T23:00:46.507ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4242024-07-19T23:00:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4252024-07-19T23:00:46.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4262024-07-19T23:00:46.508ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4272024-07-19T23:00:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4282024-07-19T23:00:46.509ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2268, tv_nsec: 705406482 } })
4292024-07-19T23:00:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4302024-07-19T23:00:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4312024-07-19T23:00:46.509ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4322024-07-19T23:00:46.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4332024-07-19T23:00:46.509ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
4342024-07-19T23:00:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4352024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4362024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4372024-07-19T23:00:46.510ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
4382024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.667208ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
4392024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4402024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4412024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4422024-07-19T23:00:46.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4432024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4442024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
4452024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 0 version = 2
4462024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4472024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4482024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4492024-07-19T23:00:46.511ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
4502024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 28.010152ms iteration = 8 status = {"physical_disks_added":0}
4512024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4522024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4532024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4542024-07-19T23:00:46.511ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4552024-07-19T23:00:46.512ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4562024-07-19T23:00:46.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4572024-07-19T23:00:46.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4582024-07-19T23:00:46.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4592024-07-19T23:00:46.513ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4602024-07-19T23:00:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
4612024-07-19T23:00:46.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4622024-07-19T23:00:46.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4632024-07-19T23:00:46.515ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4642024-07-19T23:00:46.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4652024-07-19T23:00:46.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4662024-07-19T23:00:46.515ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4672024-07-19T23:00:46.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4682024-07-19T23:00:46.516ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4692024-07-19T23:00:46.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4702024-07-19T23:00:46.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4712024-07-19T23:00:46.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4722024-07-19T23:00:46.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4732024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4742024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
4752024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
4762024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4772024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4782024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4792024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
4802024-07-19T23:00:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4812024-07-19T23:00:46.519ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 1 version = 2
4822024-07-19T23:00:46.519ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
4832024-07-19T23:00:46.519ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-07-19 22:57:48.722526 UTC dns_group = external generation = 2
4842024-07-19T23:00:46.519ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 36.65174ms iteration = 4 status = {"generation":2}
4852024-07-19T23:00:46.520ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 36.731902ms iteration = 3 status = {"nsuccess":5}
4862024-07-19T23:00:46.521ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
4872024-07-19T23:00:46.521ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.053826ms iteration = 19 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
4882024-07-19T23:00:46.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4892024-07-19T23:00:46.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4902024-07-19T23:00:46.524ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4912024-07-19T23:00:46.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4922024-07-19T23:00:46.527ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4932024-07-19T23:00:46.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4942024-07-19T23:00:46.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4952024-07-19T23:00:46.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4962024-07-19T23:00:46.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4972024-07-19T23:00:46.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4982024-07-19T23:00:46.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4992024-07-19T23:00:46.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5002024-07-19T23:00:46.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5012024-07-19T23:00:46.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5022024-07-19T23:00:46.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5032024-07-19T23:00:46.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
5042024-07-19T23:00:46.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5052024-07-19T23:00:46.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5062024-07-19T23:00:46.551ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 68.308929ms iteration = 7 status = 0
5072024-07-19T23:00:46.556ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
5082024-07-19T23:00:46.556ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
5092024-07-19T23:00:46.556ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
5102024-07-19T23:00:46.556ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5112024-07-19T23:00:46.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5122024-07-19T23:00:46.557ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5132024-07-19T23:00:46.560ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5142024-07-19T23:00:46.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5152024-07-19T23:00:46.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5162024-07-19T23:00:46.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5172024-07-19T23:00:46.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5182024-07-19T23:00:46.562ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5192024-07-19T23:00:46.562ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
5202024-07-19T23:00:46.563ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
5212024-07-19T23:00:46.563ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5222024-07-19T23:00:46.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5232024-07-19T23:00:46.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5242024-07-19T23:00:46.564ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5252024-07-19T23:00:46.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5262024-07-19T23:00:46.564ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5272024-07-19T23:00:46.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5282024-07-19T23:00:46.567ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5292024-07-19T23:00:46.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5302024-07-19T23:00:46.569ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
5312024-07-19T23:00:46.569ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
5322024-07-19T23:00:46.569ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-07-19 22:57:48.003217 UTC dns_group = internal generation = 1
5332024-07-19T23:00:46.569ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 86.578173ms iteration = 4 status = {"generation":1}
5342024-07-19T23:00:46.577ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-07-19T22:57:46.302605Z, time_modified: 2024-07-19T22:57:46.302605Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }) }
5352024-07-19T23:00:46.577ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 93.848405ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]}},"default_endpoint":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
5362024-07-19T23:00:46.651ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
5372024-07-19T23:00:46.666ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
5382024-07-19T23:00:46.697ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
5392024-07-19T23:00:46.697ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5402024-07-19T23:00:46.698ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a3042a2a-5d9d-499e-b590-113e875ebe5d", "content-length": "26", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5412024-07-19T23:00:46.698ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5422024-07-19T23:00:46.699ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5432024-07-19T23:00:46.761ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
5442024-07-19T23:00:46.761ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5452024-07-19T23:00:46.763ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b0dd181f-146f-4fdd-aae1-30edc5fa375c", "content-length": "26", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5462024-07-19T23:00:46.763ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5472024-07-19T23:00:46.764ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
5482024-07-19T23:00:46.842ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5492024-07-19T23:00:46.843ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "70cb3f06-f29d-4a2e-81a6-fc1e64855ea3", "content-length": "691", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5502024-07-19T23:00:46.843ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
5512024-07-19T23:00:46.843ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
5522024-07-19T23:00:46.843ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
5532024-07-19T23:00:46.844ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 361.292012ms iteration = 7 status = {}
5542024-07-19T23:00:46.857ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
5552024-07-19T23:00:46.857ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
5562024-07-19T23:00:46.857ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5d73abc6-8878-4ccf-9a0a-fb93c0637dbf", "content-length": "26", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5572024-07-19T23:00:46.857ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
5582024-07-19T23:00:46.858ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
5592024-07-19T23:00:46.859ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
5602024-07-19T23:00:46.879ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
5612024-07-19T23:00:46.880ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "fa5904c0-95c9-4680-9d41-4e89febd5b4b", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5622024-07-19T23:00:46.880ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5632024-07-19T23:00:46.881ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4313ddfa-3a2c-4f30-87be-83cddfef919f", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5642024-07-19T23:00:46.881ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5652024-07-19T23:00:46.882ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5662024-07-19T23:00:46.882ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "79170e5f-1b0b-4f37-b6a0-e57d731883ba", "content-length": "372", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5672024-07-19T23:00:46.882ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5682024-07-19T23:00:46.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5692024-07-19T23:00:46.883ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
5702024-07-19T23:00:46.883ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 399.778827ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
5712024-07-19T23:00:46.884ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5722024-07-19T23:00:46.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5732024-07-19T23:00:46.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5742024-07-19T23:00:46.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5752024-07-19T23:00:46.889ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5762024-07-19T23:00:46.889ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 406.01293ms iteration = 7 status = {}
5772024-07-19T23:00:46.890ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c6df10b1-26bb-49b9-985a-1277df467093", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5782024-07-19T23:00:46.891ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 408.035577ms iteration = 7 status = {}
5792024-07-19T23:00:46.919ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
5802024-07-19T23:00:46.920ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "575f5cd2-104b-4102-9c32-6832465714b7", "content-length": "1", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
5812024-07-19T23:00:46.921ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
5822024-07-19T23:00:46.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
5832024-07-19T23:00:46.923ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 439.95775ms iteration = 7 status = null
5842024-07-19T23:00:46.948ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
5852024-07-19T23:00:46.949ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4b421f01-faba-49bb-a925-5b1841168ab7", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5862024-07-19T23:00:46.949ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5872024-07-19T23:00:46.949ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "26b0f93f-e799-45c3-9141-cfd1e7d96a0a", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5882024-07-19T23:00:46.950ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5892024-07-19T23:00:46.951ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bf5fdfbf-8df5-4538-8aaa-a71708c5e996", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5902024-07-19T23:00:46.951ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5912024-07-19T23:00:46.953ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
5922024-07-19T23:00:46.953ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 469.860508ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
5932024-07-19T23:00:47.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5942024-07-19T23:00:47.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5952024-07-19T23:00:47.009ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
5962024-07-19T23:00:47.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5972024-07-19T23:00:47.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5982024-07-19T23:00:47.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 370 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = dad40c87-69ed-44ab-b660-f8d04b999d39 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
5992024-07-19T23:00:47.018ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
6002024-07-19T23:00:47.018ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2dc31110-048c-4e05-abaa-2e35479a6ec4", "content-length": "121", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
6012024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
6022024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
6032024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
6042024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
6052024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
6062024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
6072024-07-19T23:00:47.018ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
6082024-07-19T23:00:47.018ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6092024-07-19T23:00:47.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6102024-07-19T23:00:47.020ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
6112024-07-19T23:00:47.020ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "757d2ee8-d631-4b5d-88e7-a488a2004be1", "content-length": "2", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
6122024-07-19T23:00:47.020ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
6132024-07-19T23:00:47.020ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "198c567d-1429-453f-a7da-954453071ae2", "content-length": "52", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
6142024-07-19T23:00:47.020ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
6152024-07-19T23:00:47.020ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
6162024-07-19T23:00:47.071ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7eb3af91-1062-4d28-b718-8814c960e12b", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
6172024-07-19T23:00:47.071ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
6182024-07-19T23:00:47.071ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
6192024-07-19T23:00:47.072ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "87284e5d-0207-4169-a7ea-2b64057f43a5", "content-length": "494", "date": "Fri, 19 Jul 2024 23:00:46 GMT"} })
6202024-07-19T23:00:47.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6212024-07-19T23:00:47.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6222024-07-19T23:00:47.075ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6232024-07-19T23:00:47.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6242024-07-19T23:00:47.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6252024-07-19T23:00:47.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6262024-07-19T23:00:47.079ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6272024-07-19T23:00:47.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6282024-07-19T23:00:47.086ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 602.613253ms iteration = 7 status = {}
6292024-07-19T23:00:56.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
6302024-07-19T23:00:56.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6312024-07-19T23:00:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6322024-07-19T23:00:56.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
6332024-07-19T23:00:56.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
6342024-07-19T23:00:56.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6352024-07-19T23:00:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6362024-07-19T23:00:56.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6372024-07-19T23:00:56.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6382024-07-19T23:00:56.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6392024-07-19T23:00:56.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
6402024-07-19T23:00:56.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.094877ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6412024-07-19T23:00:56.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6422024-07-19T23:00:56.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6432024-07-19T23:00:56.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6442024-07-19T23:00:56.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6452024-07-19T23:00:56.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6462024-07-19T23:00:56.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6472024-07-19T23:00:56.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
6482024-07-19T23:00:56.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6492024-07-19T23:00:56.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6502024-07-19T23:00:56.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
6512024-07-19T23:00:56.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 15.962612ms iteration = 20 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
6522024-07-19T23:00:57.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 412 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 321aa3fc-9a2e-4dc2-8ed0-70677aecc78c response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
6532024-07-19T23:01:06.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
6542024-07-19T23:01:06.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6552024-07-19T23:01:06.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6562024-07-19T23:01:06.484ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
6572024-07-19T23:01:06.484ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
6582024-07-19T23:01:06.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6592024-07-19T23:01:06.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6602024-07-19T23:01:06.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6612024-07-19T23:01:06.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6622024-07-19T23:01:06.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6632024-07-19T23:01:06.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6642024-07-19T23:01:06.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6652024-07-19T23:01:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6662024-07-19T23:01:06.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
6672024-07-19T23:01:06.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.428653ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6682024-07-19T23:01:06.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6692024-07-19T23:01:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6702024-07-19T23:01:06.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6712024-07-19T23:01:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
6722024-07-19T23:01:06.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6732024-07-19T23:01:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6742024-07-19T23:01:06.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
6752024-07-19T23:01:06.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.195284ms iteration = 21 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
6762024-07-19T23:01:07.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 415 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 751c187c-39bb-4acc-9acb-99c3b662897c response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
6772024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 8 reason = Timeout
6782024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
6792024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6802024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6812024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
6822024-07-19T23:01:16.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
6832024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
6842024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6852024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
6862024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6872024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6882024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6892024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6902024-07-19T23:01:16.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6912024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
6922024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6932024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
6942024-07-19T23:01:16.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
6952024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
6962024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6972024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
6982024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6992024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7002024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
7012024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
7022024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7032024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
7042024-07-19T23:01:16.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
7052024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7062024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7072024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7082024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7092024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7102024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7112024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7122024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7132024-07-19T23:01:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7142024-07-19T23:01:16.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7152024-07-19T23:01:16.484ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7162024-07-19T23:01:16.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7172024-07-19T23:01:16.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2298, tv_nsec: 681950603 } })
7182024-07-19T23:01:16.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7192024-07-19T23:01:16.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2298, tv_nsec: 682072386 } })
7202024-07-19T23:01:16.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7212024-07-19T23:01:16.486ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
7222024-07-19T23:01:16.486ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.771849ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7232024-07-19T23:01:16.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7242024-07-19T23:01:16.486ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7252024-07-19T23:01:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
7262024-07-19T23:01:16.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
7272024-07-19T23:01:16.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.997744ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
7282024-07-19T23:01:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
7292024-07-19T23:01:16.487ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7302024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7312024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7322024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7332024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
7342024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7352024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7362024-07-19T23:01:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7372024-07-19T23:01:16.489ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
7382024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.054751ms iteration = 8 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
7392024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
7402024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7412024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7422024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7432024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7442024-07-19T23:01:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7452024-07-19T23:01:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
7462024-07-19T23:01:16.490ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7472024-07-19T23:01:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7482024-07-19T23:01:16.490ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7492024-07-19T23:01:16.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7502024-07-19T23:01:16.491ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
7512024-07-19T23:01:16.491ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7522024-07-19T23:01:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7532024-07-19T23:01:16.491ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7542024-07-19T23:01:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7552024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7562024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
7572024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7582024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7592024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7602024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7612024-07-19T23:01:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7622024-07-19T23:01:16.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
7632024-07-19T23:01:16.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7642024-07-19T23:01:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7652024-07-19T23:01:16.494ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7662024-07-19T23:01:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7672024-07-19T23:01:16.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7682024-07-19T23:01:16.495ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7692024-07-19T23:01:16.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7702024-07-19T23:01:16.496ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2298, tv_nsec: 692281492 } })
7712024-07-19T23:01:16.496ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
7722024-07-19T23:01:16.496ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.746413ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
7732024-07-19T23:01:16.496ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
7742024-07-19T23:01:16.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
7752024-07-19T23:01:16.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7762024-07-19T23:01:16.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7772024-07-19T23:01:16.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7782024-07-19T23:01:16.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7792024-07-19T23:01:16.505ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
7802024-07-19T23:01:16.505ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.473639ms iteration = 22 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
7812024-07-19T23:01:16.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7822024-07-19T23:01:16.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7832024-07-19T23:01:16.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7842024-07-19T23:01:16.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7852024-07-19T23:01:16.509ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7862024-07-19T23:01:16.511ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7872024-07-19T23:01:16.511ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7882024-07-19T23:01:16.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7892024-07-19T23:01:16.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7902024-07-19T23:01:16.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7912024-07-19T23:01:16.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7922024-07-19T23:01:16.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7932024-07-19T23:01:16.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7942024-07-19T23:01:16.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
7952024-07-19T23:01:16.516ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7962024-07-19T23:01:16.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7972024-07-19T23:01:16.517ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 34.415929ms iteration = 8 status = 0
7982024-07-19T23:01:16.525ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
7992024-07-19T23:01:16.525ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 43.508756ms iteration = 9 status = {"physical_disks_added":0}
8002024-07-19T23:01:16.609ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
8012024-07-19T23:01:16.609ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8022024-07-19T23:01:16.609ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4af1b387-c616-4dcb-9670-6ac0559ef8c7", "content-length": "26", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8032024-07-19T23:01:16.609ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8042024-07-19T23:01:16.611ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8052024-07-19T23:01:16.661ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
8062024-07-19T23:01:16.663ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "db6c55e3-357a-4983-9b94-2e92556efd47", "content-length": "691", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8072024-07-19T23:01:16.663ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
8082024-07-19T23:01:16.663ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
8092024-07-19T23:01:16.663ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
8102024-07-19T23:01:16.664ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 181.151001ms iteration = 8 status = {}
8112024-07-19T23:01:16.738ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
8122024-07-19T23:01:16.739ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "762e7e89-35c8-4ddd-9e0c-a6d2726fc799", "content-length": "1", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8132024-07-19T23:01:16.739ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
8142024-07-19T23:01:16.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
8152024-07-19T23:01:16.741ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 259.041195ms iteration = 8 status = null
8162024-07-19T23:01:16.790ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
8172024-07-19T23:01:16.790ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8182024-07-19T23:01:16.790ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "b5e86cd0-d4be-4628-9ad0-0d0947dc0473", "content-length": "26", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8192024-07-19T23:01:16.790ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8202024-07-19T23:01:16.793ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
8212024-07-19T23:01:16.807ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
8222024-07-19T23:01:16.807ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8232024-07-19T23:01:16.807ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c11d8b68-521d-4281-97a7-af176d31f83c", "content-length": "26", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8242024-07-19T23:01:16.807ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
8252024-07-19T23:01:16.809ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
8262024-07-19T23:01:16.812ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8272024-07-19T23:01:16.812ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "99ceb518-6cc2-42ed-93eb-b4cf93ba538b", "content-length": "372", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8282024-07-19T23:01:16.812ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8292024-07-19T23:01:16.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8302024-07-19T23:01:16.813ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8312024-07-19T23:01:16.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8322024-07-19T23:01:16.816ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8332024-07-19T23:01:16.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8342024-07-19T23:01:16.819ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8352024-07-19T23:01:16.820ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d8d52e36-75b0-488a-aaa2-7f9f8f19f7fb", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8362024-07-19T23:01:16.821ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 337.988646ms iteration = 8 status = {}
8372024-07-19T23:01:16.862ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 379.658958ms iteration = 8 status = {}
8382024-07-19T23:01:16.928ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8392024-07-19T23:01:16.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8402024-07-19T23:01:16.931ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
8412024-07-19T23:01:16.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422024-07-19T23:01:16.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8432024-07-19T23:01:16.940ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
8442024-07-19T23:01:16.940ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "c27d06f5-e0a0-4cd6-a8ab-1d5b0179fb65", "content-length": "121", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8452024-07-19T23:01:16.940ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
8462024-07-19T23:01:16.940ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
8472024-07-19T23:01:16.940ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
8482024-07-19T23:01:16.940ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
8492024-07-19T23:01:16.940ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
8502024-07-19T23:01:16.941ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
8512024-07-19T23:01:16.941ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
8522024-07-19T23:01:16.941ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8532024-07-19T23:01:16.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8542024-07-19T23:01:16.942ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
8552024-07-19T23:01:16.942ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "fb1754b8-536d-486b-82a2-5b2a842928ba", "content-length": "2", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8562024-07-19T23:01:16.942ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
8572024-07-19T23:01:16.942ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "795a6496-2554-420f-b60b-302fe55516be", "content-length": "52", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8582024-07-19T23:01:16.942ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
8592024-07-19T23:01:16.942ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8602024-07-19T23:01:16.981ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
8612024-07-19T23:01:16.981ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
8622024-07-19T23:01:17.012ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
8632024-07-19T23:01:17.012ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:62542
8642024-07-19T23:01:17.012ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7500f1fb-89ec-4e67-b967-67daa19d34e3", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8652024-07-19T23:01:17.012ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
8662024-07-19T23:01:17.012ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8672024-07-19T23:01:17.012ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62542 req_id = 01cd9b51-24e2-40a0-b455-8e866e3b1630 roles = RoleSet { roles: {} } uri = /metrics/producers
8682024-07-19T23:01:17.012ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62542 req_id = 01cd9b51-24e2-40a0-b455-8e866e3b1630 resource = Database result = Ok(()) uri = /metrics/producers
8692024-07-19T23:01:17.013ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "daf762b0-5461-4513-909f-417e801a434f", "content-length": "494", "date": "Fri, 19 Jul 2024 23:01:16 GMT"} })
8702024-07-19T23:01:17.013ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8712024-07-19T23:01:17.013ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 1240 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 7721eacf-76d4-4f10-b3c3-c5bd5c7ddb4d response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
8722024-07-19T23:01:17.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8732024-07-19T23:01:17.022ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8742024-07-19T23:01:17.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8752024-07-19T23:01:17.028ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8762024-07-19T23:01:17.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8772024-07-19T23:01:17.029ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8782024-07-19T23:01:17.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8792024-07-19T23:01:17.036ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 552.62011ms iteration = 8 status = {}
8802024-07-19T23:01:17.046ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = b9023b0a-a899-4caf-80bc-260e9a17ac56
8812024-07-19T23:01:17.046ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62542 req_id = 01cd9b51-24e2-40a0-b455-8e866e3b1630 roles = RoleSet { roles: {} } uri = /metrics/producers
8822024-07-19T23:01:17.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62542 req_id = 01cd9b51-24e2-40a0-b455-8e866e3b1630 resource = Database result = Ok(()) uri = /metrics/producers
8832024-07-19T23:01:17.052ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = b9023b0a-a899-4caf-80bc-260e9a17ac56 uri = http://[fd00:1122:3344:101::d]:12223/producers
8842024-07-19T23:01:17.053ZDEBGnexus: client response oximeter-collector = b9023b0a-a899-4caf-80bc-260e9a17ac56 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "504fc411-2483-4c21-98dd-6ccc1e29dc10", "date": "Fri, 19 Jul 2024 23:01:17 GMT"} })
8852024-07-19T23:01:17.053ZINFOnexus: assigned collector to new producer collector_id = b9023b0a-a899-4caf-80bc-260e9a17ac56 file = nexus/src/app/oximeter.rs:125 producer_id = a88a8918-a4d7-44fb-8a6a-ae496052bdd5
8862024-07-19T23:01:17.054ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 42270 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:62542 req_id = 01cd9b51-24e2-40a0-b455-8e866e3b1630 response_code = 201 uri = /metrics/producers
8872024-07-19T23:01:17.054ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "01cd9b51-24e2-40a0-b455-8e866e3b1630", "content-length": "41", "date": "Fri, 19 Jul 2024 23:01:17 GMT"} })
8882024-07-19T23:01:17.056ZDEBGnexus (producer-registration-task): registered with nexus successfully
8892024-07-19T23:01:17.056ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
8902024-07-19T23:01:26.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
8912024-07-19T23:01:26.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8922024-07-19T23:01:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8932024-07-19T23:01:26.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
8942024-07-19T23:01:26.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
8952024-07-19T23:01:26.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8962024-07-19T23:01:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8972024-07-19T23:01:26.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8982024-07-19T23:01:26.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8992024-07-19T23:01:26.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9002024-07-19T23:01:26.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
9012024-07-19T23:01:26.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.612736ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9022024-07-19T23:01:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9032024-07-19T23:01:26.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9042024-07-19T23:01:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9052024-07-19T23:01:26.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9062024-07-19T23:01:26.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9072024-07-19T23:01:26.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9082024-07-19T23:01:26.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
9092024-07-19T23:01:26.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9102024-07-19T23:01:26.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9112024-07-19T23:01:26.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
9122024-07-19T23:01:26.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.385999ms iteration = 23 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
9132024-07-19T23:01:27.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 355 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = b5d14ae5-54d8-4fcd-b10f-de5b4e6c6998 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
9142024-07-19T23:01:36.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
9152024-07-19T23:01:36.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9162024-07-19T23:01:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9172024-07-19T23:01:36.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
9182024-07-19T23:01:36.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
9192024-07-19T23:01:36.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9202024-07-19T23:01:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9212024-07-19T23:01:36.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9222024-07-19T23:01:36.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9232024-07-19T23:01:36.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9242024-07-19T23:01:36.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
9252024-07-19T23:01:36.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.176328ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9262024-07-19T23:01:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9272024-07-19T23:01:36.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9282024-07-19T23:01:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9292024-07-19T23:01:36.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9302024-07-19T23:01:36.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9312024-07-19T23:01:36.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9322024-07-19T23:01:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
9332024-07-19T23:01:36.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9342024-07-19T23:01:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9352024-07-19T23:01:36.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
9362024-07-19T23:01:36.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.43341ms iteration = 24 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
9372024-07-19T23:01:37.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 331 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 00717dcd-1f02-4f88-8f56-7c000bf4d3ca response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
9382024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
9392024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9402024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9412024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
9422024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9432024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
9442024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9452024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9462024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
9472024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
9482024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
9492024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9502024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
9512024-07-19T23:01:46.482ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:69 reason = blueprint disabled target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3
9522024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
9532024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 21.48\u{b5}s iteration = 6 status = {"error":"blueprint disabled","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3"}
9542024-07-19T23:01:46.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
9552024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
9562024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
9572024-07-19T23:01:46.482ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-07-19 22:51:46.482282115 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9582024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
9592024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9602024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
9612024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9622024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 9 reason = Timeout
9632024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9642024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
9652024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9662024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9672024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9682024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9692024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9702024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9712024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
9722024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
9732024-07-19T23:01:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9742024-07-19T23:01:46.482ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0f5ab007-5c53-4a1d-880a-afbba1f1b219.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 2328, tv_nsec: 679113722 } })
9752024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
9762024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
9772024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
9782024-07-19T23:01:46.483ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
9792024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9802024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
9812024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9822024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
9832024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9842024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
9852024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
9862024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9872024-07-19T23:01:46.483ZINFOnexus (BackgroundTasks): lookup region port task started background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:56
9882024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
9892024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
9902024-07-19T23:01:46.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
9912024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9922024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
9932024-07-19T23:01:46.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
9942024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9952024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
9962024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9972024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9982024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
9992024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10002024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10012024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
10022024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10032024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10042024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10052024-07-19T23:01:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10062024-07-19T23:01:46.484ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
10072024-07-19T23:01:46.484ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.944256ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
10082024-07-19T23:01:46.484ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d7897bb-e108-4529-b11c-7e55a9a521fc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4ec1dd64-fc59-46ad-8f12-dcafd769aba1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 2328, tv_nsec: 680403725 } })
10092024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10102024-07-19T23:01:46.485ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
10112024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.022195ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10122024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10132024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10142024-07-19T23:01:46.485ZINFOnexus (BackgroundTasks): lookup region port task done background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:150
10152024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.516906ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
10162024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10172024-07-19T23:01:46.485ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-07-19 22:51:46.482282115 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10182024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.606465ms iteration = 5 status = {"expiration":"2024-07-19T22:51:46.482282115Z","pruned":[]}
10192024-07-19T23:01:46.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10202024-07-19T23:01:46.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2328, tv_nsec: 682316111 } })
10212024-07-19T23:01:46.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10222024-07-19T23:01:46.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10232024-07-19T23:01:46.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10242024-07-19T23:01:46.487ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10252024-07-19T23:01:46.487ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
10262024-07-19T23:01:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 5.134154ms iteration = 5 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
10272024-07-19T23:01:46.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2328, tv_nsec: 683506332 } })
10282024-07-19T23:01:46.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
10292024-07-19T23:01:46.487ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10302024-07-19T23:01:46.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10312024-07-19T23:01:46.488ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10322024-07-19T23:01:46.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
10332024-07-19T23:01:46.489ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
10342024-07-19T23:01:46.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 6.606891ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10352024-07-19T23:01:46.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10362024-07-19T23:01:46.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10372024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10382024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10392024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10402024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10412024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10422024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10432024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10442024-07-19T23:01:46.490ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10452024-07-19T23:01:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10462024-07-19T23:01:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10472024-07-19T23:01:46.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10482024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10492024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10502024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10512024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10522024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10532024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10542024-07-19T23:01:46.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10552024-07-19T23:01:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10562024-07-19T23:01:46.493ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10572024-07-19T23:01:46.494ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
10582024-07-19T23:01:46.494ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.861478ms iteration = 9 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
10592024-07-19T23:01:46.494ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
10602024-07-19T23:01:46.494ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10612024-07-19T23:01:46.494ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10622024-07-19T23:01:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10632024-07-19T23:01:46.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10642024-07-19T23:01:46.495ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10652024-07-19T23:01:46.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10662024-07-19T23:01:46.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10672024-07-19T23:01:46.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10682024-07-19T23:01:46.495ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10692024-07-19T23:01:46.496ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
10702024-07-19T23:01:46.497ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
10712024-07-19T23:01:46.497ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
10722024-07-19T23:01:46.497ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10732024-07-19T23:01:46.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10742024-07-19T23:01:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10752024-07-19T23:01:46.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10762024-07-19T23:01:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10772024-07-19T23:01:46.498ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10782024-07-19T23:01:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10792024-07-19T23:01:46.498ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10802024-07-19T23:01:46.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10812024-07-19T23:01:46.499ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10822024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10832024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10842024-07-19T23:01:46.500ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
10852024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 17.276486ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
10862024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10872024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10882024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10892024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10902024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10912024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10922024-07-19T23:01:46.500ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2328, tv_nsec: 696784215 } })
10932024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10942024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10952024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10962024-07-19T23:01:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10972024-07-19T23:01:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10982024-07-19T23:01:46.501ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10992024-07-19T23:01:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11002024-07-19T23:01:46.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11012024-07-19T23:01:46.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11022024-07-19T23:01:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11032024-07-19T23:01:46.502ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
11042024-07-19T23:01:46.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11052024-07-19T23:01:46.502ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11062024-07-19T23:01:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11072024-07-19T23:01:46.504ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11082024-07-19T23:01:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11092024-07-19T23:01:46.504ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11102024-07-19T23:01:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11112024-07-19T23:01:46.506ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
11122024-07-19T23:01:46.506ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 0 version = 2
11132024-07-19T23:01:46.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11142024-07-19T23:01:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11152024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11162024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11172024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
11182024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11192024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11202024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11212024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
11222024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
11232024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11242024-07-19T23:01:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
11252024-07-19T23:01:46.508ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
11262024-07-19T23:01:46.508ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
11272024-07-19T23:01:46.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11282024-07-19T23:01:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11292024-07-19T23:01:46.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11302024-07-19T23:01:46.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
11312024-07-19T23:01:46.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
11322024-07-19T23:01:46.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
11332024-07-19T23:01:46.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11342024-07-19T23:01:46.513ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11352024-07-19T23:01:46.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11362024-07-19T23:01:46.515ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 1 version = 2
11372024-07-19T23:01:46.515ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
11382024-07-19T23:01:46.515ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-07-19 22:57:48.722526 UTC dns_group = external generation = 2
11392024-07-19T23:01:46.515ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 33.681305ms iteration = 5 status = {"generation":2}
11402024-07-19T23:01:46.516ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
11412024-07-19T23:01:46.516ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 34.272246ms iteration = 25 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
11422024-07-19T23:01:46.527ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
11432024-07-19T23:01:46.527ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
11442024-07-19T23:01:46.527ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-07-19 22:57:48.003217 UTC dns_group = internal generation = 1
11452024-07-19T23:01:46.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 45.057794ms iteration = 5 status = {"generation":1}
11462024-07-19T23:01:46.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11472024-07-19T23:01:46.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11482024-07-19T23:01:46.539ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
11492024-07-19T23:01:46.539ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
11502024-07-19T23:01:46.539ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 56.535754ms iteration = 5 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
11512024-07-19T23:01:46.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11522024-07-19T23:01:46.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11532024-07-19T23:01:46.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11542024-07-19T23:01:46.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
11552024-07-19T23:01:46.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11562024-07-19T23:01:46.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11572024-07-19T23:01:46.545ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 62.621604ms iteration = 9 status = 0
11582024-07-19T23:01:46.556ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
11592024-07-19T23:01:46.556ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 73.919491ms iteration = 10 status = {"physical_disks_added":0}
11602024-07-19T23:01:46.557ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11612024-07-19T23:01:46.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11622024-07-19T23:01:46.559ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11632024-07-19T23:01:46.561ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11642024-07-19T23:01:46.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11652024-07-19T23:01:46.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11662024-07-19T23:01:46.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11672024-07-19T23:01:46.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11682024-07-19T23:01:46.564ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11692024-07-19T23:01:46.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11702024-07-19T23:01:46.572ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-07-19T22:57:46.302605Z, time_modified: 2024-07-19T22:57:46.302605Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }) }
11712024-07-19T23:01:46.572ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 90.215089ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]}},"default_endpoint":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
11722024-07-19T23:01:46.665ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11732024-07-19T23:01:46.671ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
11742024-07-19T23:01:46.671ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11752024-07-19T23:01:46.671ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9bcce74b-7550-4395-abdc-7c16d863134f", "content-length": "26", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
11762024-07-19T23:01:46.671ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11772024-07-19T23:01:46.672ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11782024-07-19T23:01:46.673ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "88d67215-8f36-4d82-9b2d-54e6ab3fb7b6", "content-length": "372", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
11792024-07-19T23:01:46.673ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11802024-07-19T23:01:46.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11812024-07-19T23:01:46.674ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
11822024-07-19T23:01:46.675ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11832024-07-19T23:01:46.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11842024-07-19T23:01:46.677ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11852024-07-19T23:01:46.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11862024-07-19T23:01:46.678ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11872024-07-19T23:01:46.680ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11882024-07-19T23:01:46.680ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6bf6ce15-fd07-4fc3-b264-18556f17d5a4", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
11892024-07-19T23:01:46.681ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 198.244483ms iteration = 9 status = {}
11902024-07-19T23:01:46.794ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
11912024-07-19T23:01:46.794ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11922024-07-19T23:01:46.794ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "40fce18b-2a05-4ba1-9f5a-0641e4cce2ca", "content-length": "26", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
11932024-07-19T23:01:46.794ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11942024-07-19T23:01:46.796ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11952024-07-19T23:01:46.819ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
11962024-07-19T23:01:46.819ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11972024-07-19T23:01:46.819ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ec139e2-d50e-4774-ab30-abbb3b582e88", "content-length": "26", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
11982024-07-19T23:01:46.819ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11992024-07-19T23:01:46.820ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
12002024-07-19T23:01:46.862ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12012024-07-19T23:01:46.869ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12022024-07-19T23:01:46.869ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7a2480d0-d2ea-44a4-9992-ba3aaf9a97b5", "content-length": "691", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12032024-07-19T23:01:46.869ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
12042024-07-19T23:01:46.869ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
12052024-07-19T23:01:46.869ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
12062024-07-19T23:01:46.871ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 388.295827ms iteration = 9 status = {}
12072024-07-19T23:01:46.884ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12082024-07-19T23:01:46.885ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "be479e6a-4781-4286-b395-61abb17059be", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12092024-07-19T23:01:46.885ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12102024-07-19T23:01:46.887ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5d40debe-99bb-4cd8-98c7-a2a887ce1658", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12112024-07-19T23:01:46.887ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12122024-07-19T23:01:46.889ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12132024-07-19T23:01:46.889ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 406.927217ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
12142024-07-19T23:01:46.983ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
12152024-07-19T23:01:46.984ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bc4428e7-cca6-45f3-ad17-a02ea78d40ee", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12162024-07-19T23:01:46.984ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12172024-07-19T23:01:46.986ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "cd9214a5-5d67-4edd-9708-0394fbd409a8", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12182024-07-19T23:01:46.986ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12192024-07-19T23:01:46.986ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8de493dc-89ff-48c7-806d-1c2bee281ea5", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12202024-07-19T23:01:46.986ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12212024-07-19T23:01:46.988ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
12222024-07-19T23:01:46.988ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 506.586259ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
12232024-07-19T23:01:47.029ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
12242024-07-19T23:01:47.029ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ca9bbf90-d511-4001-b8b8-9df3b886dd3e", "content-length": "1", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12252024-07-19T23:01:47.032ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12262024-07-19T23:01:47.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12272024-07-19T23:01:47.033ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 551.422078ms iteration = 9 status = {}
12282024-07-19T23:01:47.034ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 551.703054ms iteration = 9 status = null
12292024-07-19T23:01:47.055ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12302024-07-19T23:01:47.055ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 1019 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 48f87e46-539c-47b5-bd62-b18832905420 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
12312024-07-19T23:01:47.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12322024-07-19T23:01:47.057ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
12332024-07-19T23:01:47.057ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12342024-07-19T23:01:47.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12352024-07-19T23:01:47.063ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
12362024-07-19T23:01:47.064ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7287c4ea-178c-4d0e-81a0-e54a2abdb7b9", "content-length": "121", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12372024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
12382024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
12392024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
12402024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
12412024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
12422024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
12432024-07-19T23:01:47.064ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
12442024-07-19T23:01:47.064ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12452024-07-19T23:01:47.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12462024-07-19T23:01:47.066ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
12472024-07-19T23:01:47.066ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8041fc42-c5ab-43d5-a849-f4797923c3d2", "content-length": "2", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12482024-07-19T23:01:47.066ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
12492024-07-19T23:01:47.066ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8c271720-e7b0-40f7-aa75-896ed07fee12", "content-length": "52", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12502024-07-19T23:01:47.066ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
12512024-07-19T23:01:47.066ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12522024-07-19T23:01:47.125ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "eaf597b3-aca9-47d9-a38e-4721ebd33e66", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12532024-07-19T23:01:47.125ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
12542024-07-19T23:01:47.125ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12552024-07-19T23:01:47.125ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ee1d6dee-5fd0-44c4-8d0c-b54ce920db6e", "content-length": "494", "date": "Fri, 19 Jul 2024 23:01:46 GMT"} })
12562024-07-19T23:01:47.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12572024-07-19T23:01:47.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12582024-07-19T23:01:47.129ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12592024-07-19T23:01:47.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12602024-07-19T23:01:47.132ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12612024-07-19T23:01:47.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12622024-07-19T23:01:47.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12632024-07-19T23:01:47.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12642024-07-19T23:01:47.140ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 657.528637ms iteration = 9 status = {}
12652024-07-19T23:01:56.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
12662024-07-19T23:01:56.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12672024-07-19T23:01:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12682024-07-19T23:01:56.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
12692024-07-19T23:01:56.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
12702024-07-19T23:01:56.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12712024-07-19T23:01:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12722024-07-19T23:01:56.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12732024-07-19T23:01:56.486ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12742024-07-19T23:01:56.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12752024-07-19T23:01:56.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
12762024-07-19T23:01:56.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.130135ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12772024-07-19T23:01:56.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
12782024-07-19T23:01:56.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12792024-07-19T23:01:56.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12802024-07-19T23:01:56.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12812024-07-19T23:01:56.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12822024-07-19T23:01:56.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12832024-07-19T23:01:56.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
12842024-07-19T23:01:56.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12852024-07-19T23:01:56.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12862024-07-19T23:01:56.500ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
12872024-07-19T23:01:56.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.08397ms iteration = 26 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
12882024-07-19T23:01:57.055ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 927 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 4c34e3b6-a80a-4f54-8b94-9b8a7351702c response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
12892024-07-19T23:02:06.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
12902024-07-19T23:02:06.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12912024-07-19T23:02:06.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12922024-07-19T23:02:06.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
12932024-07-19T23:02:06.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
12942024-07-19T23:02:06.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12952024-07-19T23:02:06.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12962024-07-19T23:02:06.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12972024-07-19T23:02:06.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12982024-07-19T23:02:06.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12992024-07-19T23:02:06.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
13002024-07-19T23:02:06.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.990355ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13012024-07-19T23:02:06.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13022024-07-19T23:02:06.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13032024-07-19T23:02:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13042024-07-19T23:02:06.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13052024-07-19T23:02:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13062024-07-19T23:02:06.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13072024-07-19T23:02:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
13082024-07-19T23:02:06.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13092024-07-19T23:02:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13102024-07-19T23:02:06.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
13112024-07-19T23:02:06.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.342198ms iteration = 27 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
13122024-07-19T23:02:07.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 382 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = b6b4fd5c-9e4f-4828-8653-e3e21e46b9e7 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
13132024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 10 reason = Timeout
13142024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
13152024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
13162024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
13172024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
13182024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
13192024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13202024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13212024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
13222024-07-19T23:02:16.483ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
13232024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
13242024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
13252024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13262024-07-19T23:02:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13272024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
13282024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
13292024-07-19T23:02:16.484ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
13302024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13312024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
13322024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13332024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
13342024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13352024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13362024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
13372024-07-19T23:02:16.484ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
13382024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
13392024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13402024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13412024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
13422024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13432024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13442024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13452024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13462024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13472024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
13482024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13492024-07-19T23:02:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13502024-07-19T23:02:16.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13512024-07-19T23:02:16.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2358, tv_nsec: 681552816 } })
13522024-07-19T23:02:16.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13532024-07-19T23:02:16.485ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
13542024-07-19T23:02:16.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13552024-07-19T23:02:16.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13562024-07-19T23:02:16.486ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13572024-07-19T23:02:16.487ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
13582024-07-19T23:02:16.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.112905ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
13592024-07-19T23:02:16.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13602024-07-19T23:02:16.487ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13612024-07-19T23:02:16.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2358, tv_nsec: 683769817 } })
13622024-07-19T23:02:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
13632024-07-19T23:02:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13642024-07-19T23:02:16.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13652024-07-19T23:02:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13662024-07-19T23:02:16.488ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
13672024-07-19T23:02:16.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.600644ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13682024-07-19T23:02:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13692024-07-19T23:02:16.489ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13702024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13712024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13722024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13732024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13742024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13752024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13762024-07-19T23:02:16.490ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13772024-07-19T23:02:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13782024-07-19T23:02:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13792024-07-19T23:02:16.491ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13802024-07-19T23:02:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13812024-07-19T23:02:16.492ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
13822024-07-19T23:02:16.492ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13832024-07-19T23:02:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13842024-07-19T23:02:16.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13852024-07-19T23:02:16.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13862024-07-19T23:02:16.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13872024-07-19T23:02:16.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13882024-07-19T23:02:16.494ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13892024-07-19T23:02:16.494ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13902024-07-19T23:02:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13912024-07-19T23:02:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13922024-07-19T23:02:16.494ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2358, tv_nsec: 691083531 } })
13932024-07-19T23:02:16.495ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13942024-07-19T23:02:16.495ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
13952024-07-19T23:02:16.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13962024-07-19T23:02:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13972024-07-19T23:02:16.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13982024-07-19T23:02:16.497ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13992024-07-19T23:02:16.497ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14002024-07-19T23:02:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14012024-07-19T23:02:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14022024-07-19T23:02:16.499ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
14032024-07-19T23:02:16.499ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
14042024-07-19T23:02:16.499ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 15.710667ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
14052024-07-19T23:02:16.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14062024-07-19T23:02:16.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
14072024-07-19T23:02:16.500ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
14082024-07-19T23:02:16.500ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 17.577621ms iteration = 11 status = {"physical_disks_added":0}
14092024-07-19T23:02:16.500ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14102024-07-19T23:02:16.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14112024-07-19T23:02:16.506ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
14122024-07-19T23:02:16.506ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.157973ms iteration = 28 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
14132024-07-19T23:02:16.509ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14142024-07-19T23:02:16.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14152024-07-19T23:02:16.511ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14162024-07-19T23:02:16.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14172024-07-19T23:02:16.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14182024-07-19T23:02:16.513ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14192024-07-19T23:02:16.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14202024-07-19T23:02:16.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14212024-07-19T23:02:16.514ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14222024-07-19T23:02:16.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14232024-07-19T23:02:16.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14242024-07-19T23:02:16.515ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14252024-07-19T23:02:16.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
14262024-07-19T23:02:16.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14272024-07-19T23:02:16.518ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14282024-07-19T23:02:16.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14292024-07-19T23:02:16.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14302024-07-19T23:02:16.519ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14312024-07-19T23:02:16.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
14322024-07-19T23:02:16.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14332024-07-19T23:02:16.520ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
14342024-07-19T23:02:16.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 36.225993ms iteration = 10 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
14352024-07-19T23:02:16.520ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.243712ms iteration = 10 status = 0
14362024-07-19T23:02:16.585ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
14372024-07-19T23:02:16.585ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14382024-07-19T23:02:16.585ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f7f923d3-db5e-4f30-b410-afd70a7a275a", "content-length": "26", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14392024-07-19T23:02:16.585ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14402024-07-19T23:02:16.587ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14412024-07-19T23:02:16.588ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
14422024-07-19T23:02:16.588ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14432024-07-19T23:02:16.588ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bffc1551-5ac9-4354-b566-c657104e3370", "content-length": "26", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14442024-07-19T23:02:16.588ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14452024-07-19T23:02:16.589ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
14462024-07-19T23:02:16.600ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
14472024-07-19T23:02:16.600ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14482024-07-19T23:02:16.601ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cf539302-b27f-4ac4-9e61-0e8e38189871", "content-length": "26", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14492024-07-19T23:02:16.601ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14502024-07-19T23:02:16.602ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
14512024-07-19T23:02:16.634ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14522024-07-19T23:02:16.635ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "e270dab0-19bf-4512-a7b4-a3af6e151d2a", "content-length": "372", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14532024-07-19T23:02:16.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14542024-07-19T23:02:16.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14552024-07-19T23:02:16.636ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14562024-07-19T23:02:16.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14572024-07-19T23:02:16.638ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14582024-07-19T23:02:16.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14592024-07-19T23:02:16.641ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14602024-07-19T23:02:16.641ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "c305a034-c3d7-40f9-90d1-74c933cc337b", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14612024-07-19T23:02:16.642ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 158.67822ms iteration = 10 status = {}
14622024-07-19T23:02:16.679ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
14632024-07-19T23:02:16.679ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3b5efd4e-f6fa-4f36-8a98-6cda39b0ae36", "content-length": "691", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14642024-07-19T23:02:16.679ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
14652024-07-19T23:02:16.679ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
14662024-07-19T23:02:16.679ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
14672024-07-19T23:02:16.681ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 197.5107ms iteration = 10 status = {}
14682024-07-19T23:02:16.714ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14692024-07-19T23:02:16.714ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3568ed3f-1265-49f8-94d6-9c65aa419880", "content-length": "1", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14702024-07-19T23:02:16.715ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14712024-07-19T23:02:16.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14722024-07-19T23:02:16.716ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 233.831504ms iteration = 10 status = null
14732024-07-19T23:02:16.725ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 242.203197ms iteration = 10 status = {}
14742024-07-19T23:02:16.776ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14752024-07-19T23:02:16.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14762024-07-19T23:02:16.777ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
14772024-07-19T23:02:16.777ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14782024-07-19T23:02:16.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14792024-07-19T23:02:16.784ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
14802024-07-19T23:02:16.784ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9dd619e0-8d98-4211-964c-006ae65b5acb", "content-length": "121", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14812024-07-19T23:02:16.784ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
14822024-07-19T23:02:16.784ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
14832024-07-19T23:02:16.785ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
14842024-07-19T23:02:16.785ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
14852024-07-19T23:02:16.785ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
14862024-07-19T23:02:16.785ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
14872024-07-19T23:02:16.785ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
14882024-07-19T23:02:16.785ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14892024-07-19T23:02:16.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14902024-07-19T23:02:16.786ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
14912024-07-19T23:02:16.787ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2fb4dd4d-b94b-44d4-aacb-fe9ed5b48764", "content-length": "2", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14922024-07-19T23:02:16.787ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
14932024-07-19T23:02:16.787ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "91f64b33-1e92-444f-a237-b912aacc1e05", "content-length": "52", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14942024-07-19T23:02:16.787ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
14952024-07-19T23:02:16.787ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14962024-07-19T23:02:16.840ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "11e3b85d-2d0d-4922-8d1d-c96723a41a24", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
14972024-07-19T23:02:16.840ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
14982024-07-19T23:02:16.840ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14992024-07-19T23:02:16.841ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "2cc6bb25-d9c4-4832-9dcb-d8182eeb1844", "content-length": "494", "date": "Fri, 19 Jul 2024 23:02:16 GMT"} })
15002024-07-19T23:02:16.841ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15012024-07-19T23:02:16.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15022024-07-19T23:02:16.844ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15032024-07-19T23:02:16.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15042024-07-19T23:02:16.846ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15052024-07-19T23:02:16.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15062024-07-19T23:02:16.847ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15072024-07-19T23:02:16.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15082024-07-19T23:02:16.854ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 370.827897ms iteration = 10 status = {}
15092024-07-19T23:02:17.055ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 390 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 0c4dac04-ee67-4cc3-b72c-b6976dda2fb8 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
15102024-07-19T23:02:26.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
15112024-07-19T23:02:26.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15122024-07-19T23:02:26.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15132024-07-19T23:02:26.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
15142024-07-19T23:02:26.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15152024-07-19T23:02:26.484ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
15162024-07-19T23:02:26.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15172024-07-19T23:02:26.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15182024-07-19T23:02:26.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15192024-07-19T23:02:26.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15202024-07-19T23:02:26.486ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15212024-07-19T23:02:26.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15222024-07-19T23:02:26.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15232024-07-19T23:02:26.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
15242024-07-19T23:02:26.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.583346ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15252024-07-19T23:02:26.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15262024-07-19T23:02:26.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15272024-07-19T23:02:26.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15282024-07-19T23:02:26.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
15292024-07-19T23:02:26.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15302024-07-19T23:02:26.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15312024-07-19T23:02:26.505ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
15322024-07-19T23:02:26.505ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.95885ms iteration = 29 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
15332024-07-19T23:02:27.055ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 599 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 740b7f21-f402-40ee-9280-9d360dfcc7ef response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
15342024-07-19T23:02:36.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
15352024-07-19T23:02:36.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15362024-07-19T23:02:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15372024-07-19T23:02:36.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
15382024-07-19T23:02:36.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
15392024-07-19T23:02:36.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15402024-07-19T23:02:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15412024-07-19T23:02:36.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15422024-07-19T23:02:36.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15432024-07-19T23:02:36.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15442024-07-19T23:02:36.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
15452024-07-19T23:02:36.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.162598ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15462024-07-19T23:02:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15472024-07-19T23:02:36.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15482024-07-19T23:02:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15492024-07-19T23:02:36.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15502024-07-19T23:02:36.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15512024-07-19T23:02:36.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15522024-07-19T23:02:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
15532024-07-19T23:02:36.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15542024-07-19T23:02:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15552024-07-19T23:02:36.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
15562024-07-19T23:02:36.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.855908ms iteration = 30 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
15572024-07-19T23:02:37.055ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 1098 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 52cfd373-e9f5-41cc-9a64-d5be74b1ae1f response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
15582024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
15592024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
15602024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
15612024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
15622024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
15632024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15642024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
15652024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
15662024-07-19T23:02:46.482ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-07-19 22:52:46.482802854 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
15672024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
15682024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15692024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
15702024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
15712024-07-19T23:02:46.482ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:69 reason = blueprint disabled target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3
15722024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15732024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 12.52\u{b5}s iteration = 7 status = {"error":"blueprint disabled","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3"}
15742024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
15752024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 11 reason = Timeout
15762024-07-19T23:02:46.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
15772024-07-19T23:02:46.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
15782024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
15792024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
15802024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15812024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
15822024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
15832024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
15842024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15852024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
15862024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
15872024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
15882024-07-19T23:02:46.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
15892024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15902024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15912024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
15922024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
15932024-07-19T23:02:46.483ZINFOnexus (BackgroundTasks): lookup region port task started background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:56
15942024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
15952024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
15962024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
15972024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15982024-07-19T23:02:46.483ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
15992024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16002024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16012024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
16022024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16032024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
16042024-07-19T23:02:46.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
16052024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16062024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
16072024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16082024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
16092024-07-19T23:02:46.483ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
16102024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
16112024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16122024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16132024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16142024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16152024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16162024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
16172024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16182024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16192024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16202024-07-19T23:02:46.483ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0f5ab007-5c53-4a1d-880a-afbba1f1b219.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 2388, tv_nsec: 679719733 } })
16212024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16222024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
16232024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16242024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16252024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
16262024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16272024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16282024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16292024-07-19T23:02:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16302024-07-19T23:02:46.483ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d7897bb-e108-4529-b11c-7e55a9a521fc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4ec1dd64-fc59-46ad-8f12-dcafd769aba1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 2388, tv_nsec: 679912656 } })
16312024-07-19T23:02:46.484ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16322024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16332024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16342024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16352024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
16362024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.822642ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
16372024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16382024-07-19T23:02:46.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16392024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16402024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16412024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16422024-07-19T23:02:46.486ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-07-19 22:52:46.482802854 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
16432024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.689887ms iteration = 6 status = {"expiration":"2024-07-19T22:52:46.482802854Z","pruned":[]}
16442024-07-19T23:02:46.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2388, tv_nsec: 682936842 } })
16452024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
16462024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.889691ms iteration = 6 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
16472024-07-19T23:02:46.486ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
16482024-07-19T23:02:46.486ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.104405ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
16492024-07-19T23:02:46.487ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16502024-07-19T23:02:46.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16512024-07-19T23:02:46.487ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16522024-07-19T23:02:46.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16532024-07-19T23:02:46.488ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
16542024-07-19T23:02:46.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16552024-07-19T23:02:46.488ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16562024-07-19T23:02:46.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16572024-07-19T23:02:46.488ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2388, tv_nsec: 684990119 } })
16582024-07-19T23:02:46.489ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
16592024-07-19T23:02:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16602024-07-19T23:02:46.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16612024-07-19T23:02:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16622024-07-19T23:02:46.490ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16632024-07-19T23:02:46.490ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16642024-07-19T23:02:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
16652024-07-19T23:02:46.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16662024-07-19T23:02:46.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16672024-07-19T23:02:46.495ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16682024-07-19T23:02:46.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16692024-07-19T23:02:46.497ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
16702024-07-19T23:02:46.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 14.699329ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16712024-07-19T23:02:46.497ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
16722024-07-19T23:02:46.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16732024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
16742024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
16752024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
16762024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16772024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16782024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16792024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16802024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16812024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
16822024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
16832024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
16842024-07-19T23:02:46.498ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16852024-07-19T23:02:46.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16862024-07-19T23:02:46.499ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16872024-07-19T23:02:46.499ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16882024-07-19T23:02:46.499ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16892024-07-19T23:02:46.500ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
16902024-07-19T23:02:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
16912024-07-19T23:02:46.500ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16922024-07-19T23:02:46.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16932024-07-19T23:02:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16942024-07-19T23:02:46.501ZINFOnexus (BackgroundTasks): lookup region port task done background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:150
16952024-07-19T23:02:46.501ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.372236ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
16962024-07-19T23:02:46.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16972024-07-19T23:02:46.501ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16982024-07-19T23:02:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16992024-07-19T23:02:46.502ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17002024-07-19T23:02:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17012024-07-19T23:02:46.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17022024-07-19T23:02:46.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17032024-07-19T23:02:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17042024-07-19T23:02:46.502ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17052024-07-19T23:02:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17062024-07-19T23:02:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17072024-07-19T23:02:46.503ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17082024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17092024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17102024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17112024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17122024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17132024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17142024-07-19T23:02:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17152024-07-19T23:02:46.505ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17162024-07-19T23:02:46.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17172024-07-19T23:02:46.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17182024-07-19T23:02:46.505ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
17192024-07-19T23:02:46.505ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 22.305538ms iteration = 11 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
17202024-07-19T23:02:46.505ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2388, tv_nsec: 701928259 } })
17212024-07-19T23:02:46.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
17222024-07-19T23:02:46.506ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17232024-07-19T23:02:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17242024-07-19T23:02:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17252024-07-19T23:02:46.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17262024-07-19T23:02:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17272024-07-19T23:02:46.506ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17282024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17292024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17302024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17312024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17322024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17332024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17342024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17352024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
17362024-07-19T23:02:46.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17372024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17382024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17392024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
17402024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 0 version = 2
17412024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17422024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
17432024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
17442024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17452024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17462024-07-19T23:02:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17472024-07-19T23:02:46.509ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17482024-07-19T23:02:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17492024-07-19T23:02:46.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17502024-07-19T23:02:46.511ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
17512024-07-19T23:02:46.511ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 28.118044ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
17522024-07-19T23:02:46.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
17532024-07-19T23:02:46.513ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17542024-07-19T23:02:46.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17552024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17562024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17572024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17582024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17592024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17602024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17612024-07-19T23:02:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17622024-07-19T23:02:46.515ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
17632024-07-19T23:02:46.515ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.420002ms iteration = 31 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
17642024-07-19T23:02:46.515ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17652024-07-19T23:02:46.516ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17662024-07-19T23:02:46.516ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 1 version = 2
17672024-07-19T23:02:46.516ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
17682024-07-19T23:02:46.516ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-07-19 22:57:48.722526 UTC dns_group = external generation = 2
17692024-07-19T23:02:46.516ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 33.396121ms iteration = 6 status = {"generation":2}
17702024-07-19T23:02:46.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17712024-07-19T23:02:46.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17722024-07-19T23:02:46.519ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17732024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17742024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17752024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17762024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17772024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17782024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
17792024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17802024-07-19T23:02:46.520ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17812024-07-19T23:02:46.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17822024-07-19T23:02:46.521ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17832024-07-19T23:02:46.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17842024-07-19T23:02:46.522ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17852024-07-19T23:02:46.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17862024-07-19T23:02:46.523ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
17872024-07-19T23:02:46.523ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
17882024-07-19T23:02:46.523ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-07-19 22:57:48.003217 UTC dns_group = internal generation = 1
17892024-07-19T23:02:46.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 40.46492ms iteration = 6 status = {"generation":1}
17902024-07-19T23:02:46.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17912024-07-19T23:02:46.524ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
17922024-07-19T23:02:46.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
17932024-07-19T23:02:46.525ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17942024-07-19T23:02:46.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17952024-07-19T23:02:46.527ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17962024-07-19T23:02:46.527ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17972024-07-19T23:02:46.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17982024-07-19T23:02:46.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17992024-07-19T23:02:46.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002024-07-19T23:02:46.528ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18012024-07-19T23:02:46.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
18022024-07-19T23:02:46.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18032024-07-19T23:02:46.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18042024-07-19T23:02:46.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
18052024-07-19T23:02:46.532ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18062024-07-19T23:02:46.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18072024-07-19T23:02:46.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
18082024-07-19T23:02:46.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18092024-07-19T23:02:46.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18102024-07-19T23:02:46.538ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 55.718949ms iteration = 11 status = 0
18112024-07-19T23:02:46.542ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
18122024-07-19T23:02:46.542ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
18132024-07-19T23:02:46.542ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 58.864286ms iteration = 6 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
18142024-07-19T23:02:46.545ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18152024-07-19T23:02:46.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18162024-07-19T23:02:46.547ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18172024-07-19T23:02:46.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18182024-07-19T23:02:46.548ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18192024-07-19T23:02:46.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18202024-07-19T23:02:46.550ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18212024-07-19T23:02:46.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18222024-07-19T23:02:46.551ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18232024-07-19T23:02:46.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18242024-07-19T23:02:46.553ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18252024-07-19T23:02:46.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18262024-07-19T23:02:46.553ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
18272024-07-19T23:02:46.553ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 70.962697ms iteration = 12 status = {"physical_disks_added":0}
18282024-07-19T23:02:46.554ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18292024-07-19T23:02:46.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
18302024-07-19T23:02:46.558ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18312024-07-19T23:02:46.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18322024-07-19T23:02:46.559ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18332024-07-19T23:02:46.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18342024-07-19T23:02:46.561ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18352024-07-19T23:02:46.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18362024-07-19T23:02:46.565ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18372024-07-19T23:02:46.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18382024-07-19T23:02:46.569ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18392024-07-19T23:02:46.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18402024-07-19T23:02:46.570ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18412024-07-19T23:02:46.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18422024-07-19T23:02:46.571ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18432024-07-19T23:02:46.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18442024-07-19T23:02:46.572ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18452024-07-19T23:02:46.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18462024-07-19T23:02:46.576ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18472024-07-19T23:02:46.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18482024-07-19T23:02:46.577ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18492024-07-19T23:02:46.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18502024-07-19T23:02:46.578ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18512024-07-19T23:02:46.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18522024-07-19T23:02:46.579ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18532024-07-19T23:02:46.581ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-07-19T22:57:46.302605Z, time_modified: 2024-07-19T22:57:46.302605Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }) }
18542024-07-19T23:02:46.581ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 98.304997ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]}},"default_endpoint":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
18552024-07-19T23:02:46.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
18562024-07-19T23:02:46.582ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18572024-07-19T23:02:46.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18582024-07-19T23:02:46.598ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18592024-07-19T23:02:46.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18602024-07-19T23:02:46.599ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18612024-07-19T23:02:46.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18622024-07-19T23:02:46.600ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18632024-07-19T23:02:46.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18642024-07-19T23:02:46.602ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18652024-07-19T23:02:46.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18662024-07-19T23:02:46.604ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18672024-07-19T23:02:46.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18682024-07-19T23:02:46.605ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18692024-07-19T23:02:46.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18702024-07-19T23:02:46.606ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18712024-07-19T23:02:46.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18722024-07-19T23:02:46.608ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18732024-07-19T23:02:46.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18742024-07-19T23:02:46.611ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18752024-07-19T23:02:46.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18762024-07-19T23:02:46.612ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18772024-07-19T23:02:46.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18782024-07-19T23:02:46.613ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18792024-07-19T23:02:46.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18802024-07-19T23:02:46.614ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18812024-07-19T23:02:46.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
18822024-07-19T23:02:46.617ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18832024-07-19T23:02:46.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18842024-07-19T23:02:46.626ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 77955948-8295-4a5d-aa9c-1f84d3506910, kind: Service { id: a88a8918-a4d7-44fb-8a6a-ae496052bdd5 }, name: Name("nexus-a88a8918-a4d7-44fb-8a6a-ae496052bdd5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 67])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9ae2db51-b2f1-4d2b-8c71-ae763dd181b3, kind: Service { id: 93ddd36a-2693-4ba4-9ec2-cf291bed0fda }, name: Name("nexus-93ddd36a-2693-4ba4-9ec2-cf291bed0fda"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 176])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cfc79269-6ddd-4330-abd5-292365a4d676, kind: Service { id: 5531ffc0-53da-4423-b502-3170589ff9f5 }, name: Name("nexus-5531ffc0-53da-4423-b502-3170589ff9f5"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 236, 17])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 8abb98f9-a5de-45ba-a236-906767769f98, kind: Service { id: 0f5ab007-5c53-4a1d-880a-afbba1f1b219 }, name: Name("external-dns-0f5ab007-5c53-4a1d-880a-afbba1f1b219"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 197, 111])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 96c640fb-8026-4755-b8b3-b24fde8b6e07, kind: Service { id: 5b151b39-a31a-4605-b3d7-169782f4e37d }, name: Name("external-dns-5b151b39-a31a-4605-b3d7-169782f4e37d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 162])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
18852024-07-19T23:02:46.626ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18862024-07-19T23:02:46.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18872024-07-19T23:02:46.627ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18882024-07-19T23:02:46.629ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
18892024-07-19T23:02:46.629ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
18902024-07-19T23:02:46.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
18912024-07-19T23:02:46.631ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
18922024-07-19T23:02:46.631ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 8abb98f9-a5de-45ba-a236-906767769f98, kind: Service { id: 0f5ab007-5c53-4a1d-880a-afbba1f1b219 }, name: Name("external-dns-0f5ab007-5c53-4a1d-880a-afbba1f1b219"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 197, 111])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 96c640fb-8026-4755-b8b3-b24fde8b6e07, kind: Service { id: 5b151b39-a31a-4605-b3d7-169782f4e37d }, name: Name("external-dns-5b151b39-a31a-4605-b3d7-169782f4e37d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 162])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, VpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 77955948-8295-4a5d-aa9c-1f84d3506910, kind: Service { id: a88a8918-a4d7-44fb-8a6a-ae496052bdd5 }, name: Name("nexus-a88a8918-a4d7-44fb-8a6a-ae496052bdd5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 196, 67])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9ae2db51-b2f1-4d2b-8c71-ae763dd181b3, kind: Service { id: 93ddd36a-2693-4ba4-9ec2-cf291bed0fda }, name: Name("nexus-93ddd36a-2693-4ba4-9ec2-cf291bed0fda"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 238, 176])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cfc79269-6ddd-4330-abd5-292365a4d676, kind: Service { id: 5531ffc0-53da-4423-b502-3170589ff9f5 }, name: Name("nexus-5531ffc0-53da-4423-b502-3170589ff9f5"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 236, 17])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
18932024-07-19T23:02:46.631ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
18942024-07-19T23:02:46.666ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
18952024-07-19T23:02:46.666ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18962024-07-19T23:02:46.666ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7b49fc9e-63c7-46db-a583-52c4e2ccfd46", "content-length": "26", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
18972024-07-19T23:02:46.666ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18982024-07-19T23:02:46.668ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18992024-07-19T23:02:46.717ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, time_created: 2024-07-19T22:57:48.313305Z, time_modified: 2024-07-19T22:57:48.313305Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
19002024-07-19T23:02:46.717ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
19012024-07-19T23:02:46.718ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19022024-07-19T23:02:46.719ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "545dd0af-47d7-4158-a8fc-08c803f57d1e", "content-length": "372", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19032024-07-19T23:02:46.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19042024-07-19T23:02:46.719ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19052024-07-19T23:02:46.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19062024-07-19T23:02:46.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19072024-07-19T23:02:46.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19082024-07-19T23:02:46.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19092024-07-19T23:02:46.721ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
19102024-07-19T23:02:46.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
19112024-07-19T23:02:46.722ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19122024-07-19T23:02:46.722ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19132024-07-19T23:02:46.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19142024-07-19T23:02:46.725ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
19152024-07-19T23:02:46.725ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19162024-07-19T23:02:46.725ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "bce1b967-8be5-479a-ad34-9b326d2958ee", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19172024-07-19T23:02:46.727ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 244.124022ms iteration = 11 status = {}
19182024-07-19T23:02:46.756ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
19192024-07-19T23:02:46.758ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5a8ba08a-7488-41b9-a6e1-b8d1eb174734", "content-length": "691", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19202024-07-19T23:02:46.758ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
19212024-07-19T23:02:46.758ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
19222024-07-19T23:02:46.758ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
19232024-07-19T23:02:46.760ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 277.077275ms iteration = 11 status = {}
19242024-07-19T23:02:46.760ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
19252024-07-19T23:02:46.760ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19262024-07-19T23:02:46.760ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "1f9d0ea7-77d2-490d-be28-b88a2fd1cc0c", "content-length": "26", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19272024-07-19T23:02:46.760ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19282024-07-19T23:02:46.762ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
19292024-07-19T23:02:46.861ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
19302024-07-19T23:02:46.891ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19312024-07-19T23:02:46.892ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ad974996-4b9e-410d-906e-d15e654ecf0c", "content-length": "1", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19322024-07-19T23:02:46.893ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19332024-07-19T23:02:46.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19342024-07-19T23:02:46.894ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 411.666134ms iteration = 11 status = null
19352024-07-19T23:02:46.907ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19362024-07-19T23:02:46.908ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = service_firewall_rule_propagation result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", query: None, fragment: None }, status: 204, headers: {"x-request-id": "77031114-f584-429b-afee-b635efcbeb6a", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19372024-07-19T23:02:46.910ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:494
19382024-07-19T23:02:46.910ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 427.35087ms
19392024-07-19T23:02:46.910ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 427.35977ms iteration = 2 status = {}
19402024-07-19T23:02:46.913ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
19412024-07-19T23:02:46.929ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
19422024-07-19T23:02:46.929ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19432024-07-19T23:02:46.930ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7772f6bf-5ccb-41b2-ae87-f3b53a93ad7d", "content-length": "26", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19442024-07-19T23:02:46.930ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19452024-07-19T23:02:46.931ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19462024-07-19T23:02:47.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 344 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 0b98ed69-1e24-4c1f-b992-6e609eaf339b response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
19472024-07-19T23:02:47.061ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 578.91118ms iteration = 11 status = {}
19482024-07-19T23:02:47.123ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
19492024-07-19T23:02:47.123ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "44b6f93a-9ffe-4adf-9b09-636dba000c22", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19502024-07-19T23:02:47.123ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19512024-07-19T23:02:47.125ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f2a350a4-4688-40aa-818b-8ee24a2cf694", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19522024-07-19T23:02:47.125ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19532024-07-19T23:02:47.127ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19542024-07-19T23:02:47.127ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 644.960828ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
19552024-07-19T23:02:47.130ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
19562024-07-19T23:02:47.131ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19572024-07-19T23:02:47.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19582024-07-19T23:02:47.133ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
19592024-07-19T23:02:47.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19602024-07-19T23:02:47.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19612024-07-19T23:02:47.146ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19622024-07-19T23:02:47.146ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "65846b93-bf67-4b52-aee4-ab90c16a145f", "content-length": "121", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19632024-07-19T23:02:47.146ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
19642024-07-19T23:02:47.146ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
19652024-07-19T23:02:47.147ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
19662024-07-19T23:02:47.147ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
19672024-07-19T23:02:47.147ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
19682024-07-19T23:02:47.147ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
19692024-07-19T23:02:47.147ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
19702024-07-19T23:02:47.147ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19712024-07-19T23:02:47.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19722024-07-19T23:02:47.150ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19732024-07-19T23:02:47.150ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0a2a8c38-ce99-4323-8384-dd4eefb58840", "content-length": "2", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19742024-07-19T23:02:47.150ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19752024-07-19T23:02:47.150ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8279ab38-bf94-432d-889e-c34700f33e35", "content-length": "52", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} })
19762024-07-19T23:02:47.150ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
19772024-07-19T23:02:47.150ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19782024-07-19T23:02:47.163ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
19792024-07-19T23:02:47.164ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "8869d1ef-ca82-44b1-a43d-68df12afde17", "date": "Fri, 19 Jul 2024 23:02:47 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19802024-07-19T23:02:47.164ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19812024-07-19T23:02:47.165ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f9d5ce96-78b3-48ba-b533-44272822d8b9", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19822024-07-19T23:02:47.165ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19832024-07-19T23:02:47.167ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "553e8d8d-175b-438d-a994-a216b7e34878", "date": "Fri, 19 Jul 2024 23:02:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19842024-07-19T23:02:47.167ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19852024-07-19T23:02:47.169ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
19862024-07-19T23:02:47.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 686.379404ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
19872024-07-19T23:02:47.202ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "6215a8da-5195-425b-8bc1-2cbc62accbe4", "date": "Fri, 19 Jul 2024 23:02:47 GMT"} })
19882024-07-19T23:02:47.202ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
19892024-07-19T23:02:47.202ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
19902024-07-19T23:02:47.202ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "84d62ba8-5056-4669-aa6c-5bb7d53874df", "content-length": "494", "date": "Fri, 19 Jul 2024 23:02:47 GMT"} })
19912024-07-19T23:02:47.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19922024-07-19T23:02:47.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19932024-07-19T23:02:47.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19942024-07-19T23:02:47.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19952024-07-19T23:02:47.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19962024-07-19T23:02:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19972024-07-19T23:02:47.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19982024-07-19T23:02:47.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19992024-07-19T23:02:47.216ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 733.364033ms iteration = 11 status = {}
20002024-07-19T23:02:56.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
20012024-07-19T23:02:56.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20022024-07-19T23:02:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20032024-07-19T23:02:56.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
20042024-07-19T23:02:56.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
20052024-07-19T23:02:56.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20062024-07-19T23:02:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20072024-07-19T23:02:56.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20082024-07-19T23:02:56.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20092024-07-19T23:02:56.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20102024-07-19T23:02:56.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
20112024-07-19T23:02:56.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.446963ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20122024-07-19T23:02:56.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20132024-07-19T23:02:56.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20142024-07-19T23:02:56.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20152024-07-19T23:02:56.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20162024-07-19T23:02:56.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20172024-07-19T23:02:56.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20182024-07-19T23:02:56.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
20192024-07-19T23:02:56.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20202024-07-19T23:02:56.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20212024-07-19T23:02:56.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
20222024-07-19T23:02:56.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.461621ms iteration = 32 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
20232024-07-19T23:02:57.055ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 1132 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 5e76748c-e8d7-4fa3-b775-0462bdf6a0bb response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
20242024-07-19T23:03:06.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
20252024-07-19T23:03:06.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20262024-07-19T23:03:06.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20272024-07-19T23:03:06.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
20282024-07-19T23:03:06.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
20292024-07-19T23:03:06.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20302024-07-19T23:03:06.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20312024-07-19T23:03:06.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20322024-07-19T23:03:06.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20332024-07-19T23:03:06.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20342024-07-19T23:03:06.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
20352024-07-19T23:03:06.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.351102ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20362024-07-19T23:03:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20372024-07-19T23:03:06.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20382024-07-19T23:03:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20392024-07-19T23:03:06.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20402024-07-19T23:03:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20412024-07-19T23:03:06.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20422024-07-19T23:03:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
20432024-07-19T23:03:06.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20442024-07-19T23:03:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20452024-07-19T23:03:06.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
20462024-07-19T23:03:06.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.541983ms iteration = 33 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
20472024-07-19T23:03:07.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 354 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 0eebd912-9e7c-4403-8f85-82ba4345c08a response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
20482024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 12 reason = Timeout
20492024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20502024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
20512024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20522024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
20532024-07-19T23:03:16.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
20542024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
20552024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20562024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
20572024-07-19T23:03:16.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20582024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20592024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
20602024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20612024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20622024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
20632024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
20642024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
20652024-07-19T23:03:16.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
20662024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20672024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
20682024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20692024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
20702024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20712024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
20722024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
20732024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20742024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20752024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
20762024-07-19T23:03:16.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
20772024-07-19T23:03:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20782024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20792024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20802024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20812024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20822024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
20832024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
20842024-07-19T23:03:16.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
20852024-07-19T23:03:16.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20862024-07-19T23:03:16.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20872024-07-19T23:03:16.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2418, tv_nsec: 681828061 } })
20882024-07-19T23:03:16.485ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2418, tv_nsec: 682080276 } })
20892024-07-19T23:03:16.486ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20902024-07-19T23:03:16.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20912024-07-19T23:03:16.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20922024-07-19T23:03:16.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20932024-07-19T23:03:16.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20942024-07-19T23:03:16.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20952024-07-19T23:03:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
20962024-07-19T23:03:16.487ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
20972024-07-19T23:03:16.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.344789ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20982024-07-19T23:03:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
20992024-07-19T23:03:16.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
21002024-07-19T23:03:16.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.579546ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21012024-07-19T23:03:16.489ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21022024-07-19T23:03:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21032024-07-19T23:03:16.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21042024-07-19T23:03:16.489ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21052024-07-19T23:03:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21062024-07-19T23:03:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21072024-07-19T23:03:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
21082024-07-19T23:03:16.490ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21092024-07-19T23:03:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21102024-07-19T23:03:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21112024-07-19T23:03:16.490ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21122024-07-19T23:03:16.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21132024-07-19T23:03:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21142024-07-19T23:03:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21152024-07-19T23:03:16.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21162024-07-19T23:03:16.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21172024-07-19T23:03:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
21182024-07-19T23:03:16.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21192024-07-19T23:03:16.492ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
21202024-07-19T23:03:16.492ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21212024-07-19T23:03:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21222024-07-19T23:03:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21232024-07-19T23:03:16.492ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2418, tv_nsec: 688987652 } })
21242024-07-19T23:03:16.493ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
21252024-07-19T23:03:16.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21262024-07-19T23:03:16.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21272024-07-19T23:03:16.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21282024-07-19T23:03:16.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21292024-07-19T23:03:16.493ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21302024-07-19T23:03:16.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21312024-07-19T23:03:16.494ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21322024-07-19T23:03:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21332024-07-19T23:03:16.494ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21342024-07-19T23:03:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21352024-07-19T23:03:16.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21362024-07-19T23:03:16.496ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
21372024-07-19T23:03:16.496ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 12.518268ms iteration = 12 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
21382024-07-19T23:03:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21392024-07-19T23:03:16.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21402024-07-19T23:03:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21412024-07-19T23:03:16.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
21422024-07-19T23:03:16.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21432024-07-19T23:03:16.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21442024-07-19T23:03:16.499ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
21452024-07-19T23:03:16.499ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 15.254559ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
21462024-07-19T23:03:16.505ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
21472024-07-19T23:03:16.505ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.770796ms iteration = 34 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
21482024-07-19T23:03:16.508ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21492024-07-19T23:03:16.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21502024-07-19T23:03:16.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21512024-07-19T23:03:16.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21522024-07-19T23:03:16.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21532024-07-19T23:03:16.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
21542024-07-19T23:03:16.514ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21552024-07-19T23:03:16.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21562024-07-19T23:03:16.515ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21572024-07-19T23:03:16.515ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 31.454465ms iteration = 12 status = 0
21582024-07-19T23:03:16.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21592024-07-19T23:03:16.522ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21602024-07-19T23:03:16.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21612024-07-19T23:03:16.524ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21622024-07-19T23:03:16.526ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
21632024-07-19T23:03:16.526ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 43.696328ms iteration = 13 status = {"physical_disks_added":0}
21642024-07-19T23:03:16.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21652024-07-19T23:03:16.526ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21662024-07-19T23:03:16.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21672024-07-19T23:03:16.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21682024-07-19T23:03:16.529ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21692024-07-19T23:03:16.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21702024-07-19T23:03:16.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21712024-07-19T23:03:16.612ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
21722024-07-19T23:03:16.612ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21732024-07-19T23:03:16.613ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "ac700809-36ce-461e-b515-63f72405501c", "content-length": "26", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
21742024-07-19T23:03:16.613ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21752024-07-19T23:03:16.614ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21762024-07-19T23:03:16.623ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
21772024-07-19T23:03:16.623ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21782024-07-19T23:03:16.623ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "05eed7f1-aa43-4b93-8a5f-6d189198110e", "content-length": "26", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
21792024-07-19T23:03:16.623ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21802024-07-19T23:03:16.626ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
21812024-07-19T23:03:16.715ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
21822024-07-19T23:03:16.717ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4ec7ca4b-0fdf-4391-8f7b-1c615262685a", "content-length": "691", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
21832024-07-19T23:03:16.717ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
21842024-07-19T23:03:16.717ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
21852024-07-19T23:03:16.717ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
21862024-07-19T23:03:16.719ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 235.77767ms iteration = 12 status = {}
21872024-07-19T23:03:16.725ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
21882024-07-19T23:03:16.725ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21892024-07-19T23:03:16.726ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f524e716-e472-4c44-a115-8c81682e756c", "content-length": "26", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
21902024-07-19T23:03:16.726ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21912024-07-19T23:03:16.727ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21922024-07-19T23:03:16.736ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21932024-07-19T23:03:16.736ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "52168adf-8880-482f-a466-b8644e56316f", "content-length": "372", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
21942024-07-19T23:03:16.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21952024-07-19T23:03:16.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21962024-07-19T23:03:16.738ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21972024-07-19T23:03:16.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21982024-07-19T23:03:16.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21992024-07-19T23:03:16.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22002024-07-19T23:03:16.744ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22012024-07-19T23:03:16.744ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "7de8e4fa-088d-4097-bbcd-7e88f11aff28", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22022024-07-19T23:03:16.746ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 262.584069ms iteration = 12 status = {}
22032024-07-19T23:03:16.762ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
22042024-07-19T23:03:16.762ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "8c8d460e-5eeb-434e-86c6-1bec0fb6a37a", "content-length": "1", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22052024-07-19T23:03:16.764ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22062024-07-19T23:03:16.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22072024-07-19T23:03:16.765ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 282.629445ms iteration = 12 status = null
22082024-07-19T23:03:16.810ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 327.763001ms iteration = 12 status = {}
22092024-07-19T23:03:16.851ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22102024-07-19T23:03:16.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22112024-07-19T23:03:16.853ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
22122024-07-19T23:03:16.853ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22132024-07-19T23:03:16.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22142024-07-19T23:03:16.859ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
22152024-07-19T23:03:16.860ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bc8dbec4-7e08-4e47-9c04-53c9b0f5da8b", "content-length": "121", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22162024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
22172024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
22182024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
22192024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
22202024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
22212024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
22222024-07-19T23:03:16.860ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
22232024-07-19T23:03:16.860ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22242024-07-19T23:03:16.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22252024-07-19T23:03:16.862ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
22262024-07-19T23:03:16.862ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0ce53ae9-cdbc-4e5a-927b-1d508a2c33a8", "content-length": "2", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22272024-07-19T23:03:16.862ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
22282024-07-19T23:03:16.862ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "6cadcca6-5afd-45ee-a771-a701ab02a223", "content-length": "52", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22292024-07-19T23:03:16.862ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
22302024-07-19T23:03:16.862ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
22312024-07-19T23:03:16.929ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "b69ba99b-7cca-4ff5-a5e7-468774938003", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22322024-07-19T23:03:16.930ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
22332024-07-19T23:03:16.930ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
22342024-07-19T23:03:16.930ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7a450556-a4a4-4dc6-aa85-cd15bf6cd79e", "content-length": "494", "date": "Fri, 19 Jul 2024 23:03:16 GMT"} })
22352024-07-19T23:03:16.930ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22362024-07-19T23:03:16.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22372024-07-19T23:03:16.934ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22382024-07-19T23:03:16.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22392024-07-19T23:03:16.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22402024-07-19T23:03:16.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22412024-07-19T23:03:16.939ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22422024-07-19T23:03:16.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22432024-07-19T23:03:16.946ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 462.86083ms iteration = 12 status = {}
22442024-07-19T23:03:17.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 350 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 3ef87706-2cf8-4a3a-ae1a-36818518451a response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
22452024-07-19T23:03:25.601ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::1:0]:63689
22462024-07-19T23:03:25.601ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:63689 req_id = 3699f983-37fb-4dc4-a3f4-54a9c558e213 roles = RoleSet { roles: {} } uri = /metrics/producers
22472024-07-19T23:03:25.602ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:63689 req_id = 3699f983-37fb-4dc4-a3f4-54a9c558e213 resource = Database result = Ok(()) uri = /metrics/producers
22482024-07-19T23:03:25.634ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = b9023b0a-a899-4caf-80bc-260e9a17ac56
22492024-07-19T23:03:25.634ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:63689 req_id = 3699f983-37fb-4dc4-a3f4-54a9c558e213 roles = RoleSet { roles: {} } uri = /metrics/producers
22502024-07-19T23:03:25.634ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:63689 req_id = 3699f983-37fb-4dc4-a3f4-54a9c558e213 resource = Database result = Ok(()) uri = /metrics/producers
22512024-07-19T23:03:25.673ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = b9023b0a-a899-4caf-80bc-260e9a17ac56 uri = http://[fd00:1122:3344:101::d]:12223/producers
22522024-07-19T23:03:25.673ZDEBGnexus: client response oximeter-collector = b9023b0a-a899-4caf-80bc-260e9a17ac56 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d7230595-ade1-4a51-82c5-665e66f340da", "date": "Fri, 19 Jul 2024 23:03:25 GMT"} })
22532024-07-19T23:03:25.674ZINFOnexus: assigned collector to new producer collector_id = b9023b0a-a899-4caf-80bc-260e9a17ac56 file = nexus/src/app/oximeter.rs:125 producer_id = 4a43dfee-c089-4a8c-bcfe-f8b4c67bb3c2
22542024-07-19T23:03:25.676ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 74244 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::1:0]:63689 req_id = 3699f983-37fb-4dc4-a3f4-54a9c558e213 response_code = 201 uri = /metrics/producers
22552024-07-19T23:03:26.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
22562024-07-19T23:03:26.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22572024-07-19T23:03:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22582024-07-19T23:03:26.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
22592024-07-19T23:03:26.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
22602024-07-19T23:03:26.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22612024-07-19T23:03:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22622024-07-19T23:03:26.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22632024-07-19T23:03:26.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22642024-07-19T23:03:26.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22652024-07-19T23:03:26.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
22662024-07-19T23:03:26.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.620316ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22672024-07-19T23:03:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22682024-07-19T23:03:26.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22692024-07-19T23:03:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22702024-07-19T23:03:26.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22712024-07-19T23:03:26.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22722024-07-19T23:03:26.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22732024-07-19T23:03:26.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
22742024-07-19T23:03:26.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22752024-07-19T23:03:26.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22762024-07-19T23:03:26.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
22772024-07-19T23:03:26.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.694905ms iteration = 35 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
22782024-07-19T23:03:27.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 350 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = b6f2305c-7645-4ee7-aaaf-475ca83e0663 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
22792024-07-19T23:03:36.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
22802024-07-19T23:03:36.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22812024-07-19T23:03:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22822024-07-19T23:03:36.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
22832024-07-19T23:03:36.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
22842024-07-19T23:03:36.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22852024-07-19T23:03:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22862024-07-19T23:03:36.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22872024-07-19T23:03:36.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22882024-07-19T23:03:36.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22892024-07-19T23:03:36.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
22902024-07-19T23:03:36.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.058606ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22912024-07-19T23:03:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22922024-07-19T23:03:36.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22932024-07-19T23:03:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22942024-07-19T23:03:36.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22952024-07-19T23:03:36.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22962024-07-19T23:03:36.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22972024-07-19T23:03:36.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
22982024-07-19T23:03:36.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22992024-07-19T23:03:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23002024-07-19T23:03:36.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
23012024-07-19T23:03:36.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.218606ms iteration = 36 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
23022024-07-19T23:03:37.056ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 1289 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = ad582619-3c9c-4a16-88e6-0596af400fd7 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
23032024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
23042024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
23052024-07-19T23:03:46.482ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:69 reason = blueprint disabled target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3
23062024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
23072024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 25\u{b5}s iteration = 8 status = {"error":"blueprint disabled","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3"}
23082024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 13 reason = Timeout
23092024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
23102024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23112024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
23122024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
23132024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23142024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
23152024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
23162024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23172024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
23182024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
23192024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23202024-07-19T23:03:46.482ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-07-19 22:53:46.482577362 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
23212024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
23222024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
23232024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
23242024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
23252024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
23262024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23272024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
23282024-07-19T23:03:46.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
23292024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
23302024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
23312024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23322024-07-19T23:03:46.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
23332024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23342024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
23352024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23362024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23372024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23382024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
23392024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
23402024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
23412024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
23422024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
23432024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23442024-07-19T23:03:46.483ZINFOnexus (BackgroundTasks): lookup region port task started background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:56
23452024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23462024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23472024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
23482024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
23492024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
23502024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23512024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
23522024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23532024-07-19T23:03:46.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
23542024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23552024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
23562024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23572024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
23582024-07-19T23:03:46.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
23592024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
23602024-07-19T23:03:46.483ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
23612024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23622024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
23632024-07-19T23:03:46.483ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0f5ab007-5c53-4a1d-880a-afbba1f1b219.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 2448, tv_nsec: 679694284 } })
23642024-07-19T23:03:46.483ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d7897bb-e108-4529-b11c-7e55a9a521fc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4ec1dd64-fc59-46ad-8f12-dcafd769aba1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 2448, tv_nsec: 679695133 } })
23652024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23662024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
23672024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23682024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23692024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23702024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23712024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
23722024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23732024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23742024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23752024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23762024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23772024-07-19T23:03:46.483ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23782024-07-19T23:03:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23792024-07-19T23:03:46.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23802024-07-19T23:03:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23812024-07-19T23:03:46.484ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23822024-07-19T23:03:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23832024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
23842024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.422085ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
23852024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23862024-07-19T23:03:46.485ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-07-19 22:53:46.482577362 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
23872024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.621197ms iteration = 7 status = {"expiration":"2024-07-19T22:53:46.482577362Z","pruned":[]}
23882024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23892024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23902024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23912024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
23922024-07-19T23:03:46.485ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.367681ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
23932024-07-19T23:03:46.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
23942024-07-19T23:03:46.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23952024-07-19T23:03:46.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
23962024-07-19T23:03:46.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2448, tv_nsec: 683029765 } })
23972024-07-19T23:03:46.487ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
23982024-07-19T23:03:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.684416ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
23992024-07-19T23:03:46.487ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24002024-07-19T23:03:46.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
24012024-07-19T23:03:46.487ZINFOnexus (BackgroundTasks): lookup region port task done background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:150
24022024-07-19T23:03:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.487142ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
24032024-07-19T23:03:46.488ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24042024-07-19T23:03:46.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24052024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24062024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
24072024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
24082024-07-19T23:03:46.489ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2448, tv_nsec: 685259435 } })
24092024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
24102024-07-19T23:03:46.489ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
24112024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 5.870978ms iteration = 7 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
24122024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
24132024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24142024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24152024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24162024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24172024-07-19T23:03:46.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24182024-07-19T23:03:46.490ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
24192024-07-19T23:03:46.491ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
24202024-07-19T23:03:46.491ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24212024-07-19T23:03:46.491ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24222024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24232024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24242024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24252024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24262024-07-19T23:03:46.492ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
24272024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24282024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24292024-07-19T23:03:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24302024-07-19T23:03:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24312024-07-19T23:03:46.493ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24322024-07-19T23:03:46.493ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
24332024-07-19T23:03:46.493ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 10.248107ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24342024-07-19T23:03:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24352024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24362024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24372024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24382024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24392024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24402024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24412024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24422024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24432024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24442024-07-19T23:03:46.494ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24452024-07-19T23:03:46.495ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24462024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24472024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24482024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24492024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
24502024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
24512024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
24522024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
24532024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24542024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24552024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
24562024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
24572024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24582024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
24592024-07-19T23:03:46.496ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24602024-07-19T23:03:46.497ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2448, tv_nsec: 693462065 } })
24612024-07-19T23:03:46.497ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
24622024-07-19T23:03:46.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.271021ms iteration = 13 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
24632024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24642024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24652024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24662024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
24672024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24682024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24692024-07-19T23:03:46.498ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
24702024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24712024-07-19T23:03:46.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
24722024-07-19T23:03:46.500ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24732024-07-19T23:03:46.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24742024-07-19T23:03:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24752024-07-19T23:03:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
24762024-07-19T23:03:46.501ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
24772024-07-19T23:03:46.501ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
24782024-07-19T23:03:46.501ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.192391ms iteration = 14 status = {"physical_disks_added":0}
24792024-07-19T23:03:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
24802024-07-19T23:03:46.501ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24812024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
24822024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24832024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
24842024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24852024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24862024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24872024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
24882024-07-19T23:03:46.502ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24892024-07-19T23:03:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24902024-07-19T23:03:46.503ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24912024-07-19T23:03:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
24922024-07-19T23:03:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
24932024-07-19T23:03:46.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
24942024-07-19T23:03:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
24952024-07-19T23:03:46.504ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24962024-07-19T23:03:46.505ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24972024-07-19T23:03:46.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24982024-07-19T23:03:46.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
24992024-07-19T23:03:46.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25002024-07-19T23:03:46.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25012024-07-19T23:03:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
25022024-07-19T23:03:46.507ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
25032024-07-19T23:03:46.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
25042024-07-19T23:03:46.508ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
25052024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.764282ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
25062024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
25072024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25082024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
25092024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 0 version = 2
25102024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25112024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25122024-07-19T23:03:46.508ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25132024-07-19T23:03:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25142024-07-19T23:03:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25152024-07-19T23:03:46.509ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 25.763831ms iteration = 4 status = {"nsuccess":5}
25162024-07-19T23:03:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25172024-07-19T23:03:46.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25182024-07-19T23:03:46.510ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
25192024-07-19T23:03:46.510ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
25202024-07-19T23:03:46.510ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
25212024-07-19T23:03:46.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25222024-07-19T23:03:46.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25232024-07-19T23:03:46.512ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25242024-07-19T23:03:46.512ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
25252024-07-19T23:03:46.512ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.298624ms iteration = 37 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
25262024-07-19T23:03:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
25272024-07-19T23:03:46.514ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25282024-07-19T23:03:46.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25292024-07-19T23:03:46.515ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 1 version = 2
25302024-07-19T23:03:46.515ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
25312024-07-19T23:03:46.515ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-07-19 22:57:48.722526 UTC dns_group = external generation = 2
25322024-07-19T23:03:46.515ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 33.165257ms iteration = 7 status = {"generation":2}
25332024-07-19T23:03:46.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25342024-07-19T23:03:46.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25352024-07-19T23:03:46.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25362024-07-19T23:03:46.517ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25372024-07-19T23:03:46.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25382024-07-19T23:03:46.518ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
25392024-07-19T23:03:46.518ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
25402024-07-19T23:03:46.518ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25412024-07-19T23:03:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25422024-07-19T23:03:46.518ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25432024-07-19T23:03:46.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25442024-07-19T23:03:46.519ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25452024-07-19T23:03:46.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25462024-07-19T23:03:46.520ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25472024-07-19T23:03:46.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25482024-07-19T23:03:46.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25492024-07-19T23:03:46.522ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25502024-07-19T23:03:46.522ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25512024-07-19T23:03:46.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25522024-07-19T23:03:46.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25532024-07-19T23:03:46.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25542024-07-19T23:03:46.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25552024-07-19T23:03:46.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25562024-07-19T23:03:46.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
25572024-07-19T23:03:46.524ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25582024-07-19T23:03:46.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25592024-07-19T23:03:46.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25602024-07-19T23:03:46.525ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25612024-07-19T23:03:46.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25622024-07-19T23:03:46.529ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
25632024-07-19T23:03:46.529ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
25642024-07-19T23:03:46.529ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-07-19 22:57:48.003217 UTC dns_group = internal generation = 1
25652024-07-19T23:03:46.529ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 45.638744ms iteration = 7 status = {"generation":1}
25662024-07-19T23:03:46.530ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 46.936488ms iteration = 13 status = 0
25672024-07-19T23:03:46.562ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-07-19T22:57:46.302605Z, time_modified: 2024-07-19T22:57:46.302605Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
25682024-07-19T23:03:46.562ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 80.374259ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]}},"default_endpoint":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
25692024-07-19T23:03:46.658ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
25702024-07-19T23:03:46.658ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25712024-07-19T23:03:46.659ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "643f6630-7896-4842-b7d3-cfb029cb33d4", "content-length": "26", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
25722024-07-19T23:03:46.659ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25732024-07-19T23:03:46.660ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25742024-07-19T23:03:46.706ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
25752024-07-19T23:03:46.707ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a41375be-d4d6-4ba4-8cf5-8a0f2950cf19", "content-length": "691", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
25762024-07-19T23:03:46.707ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
25772024-07-19T23:03:46.707ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
25782024-07-19T23:03:46.707ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
25792024-07-19T23:03:46.708ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 225.511942ms iteration = 13 status = {}
25802024-07-19T23:03:46.722ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
25812024-07-19T23:03:46.722ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25822024-07-19T23:03:46.723ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "5aa8611a-0617-4d66-99ce-662d6b1b0663", "content-length": "26", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
25832024-07-19T23:03:46.723ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25842024-07-19T23:03:46.725ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
25852024-07-19T23:03:46.813ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
25862024-07-19T23:03:46.821ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
25872024-07-19T23:03:46.828ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
25882024-07-19T23:03:46.828ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25892024-07-19T23:03:46.829ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "acc4f9de-fa3f-4e20-a154-d0629b7abd0b", "content-length": "26", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
25902024-07-19T23:03:46.829ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25912024-07-19T23:03:46.830ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25922024-07-19T23:03:46.852ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25932024-07-19T23:03:46.852ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "07b238fc-5dd7-41c1-9200-ceb1713d6ff0", "content-length": "372", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
25942024-07-19T23:03:46.852ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25952024-07-19T23:03:46.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25962024-07-19T23:03:46.854ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25972024-07-19T23:03:46.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25982024-07-19T23:03:46.856ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25992024-07-19T23:03:46.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26002024-07-19T23:03:46.859ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
26012024-07-19T23:03:46.860ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "3b5933f3-f310-4086-a93d-60e813a1f2a7", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
26022024-07-19T23:03:46.862ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 378.785512ms iteration = 13 status = {}
26032024-07-19T23:03:46.872ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
26042024-07-19T23:03:46.872ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "0d64c0c4-7141-415f-804b-7afc313ec5f8", "content-length": "2", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
26052024-07-19T23:03:46.874ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
26062024-07-19T23:03:46.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
26072024-07-19T23:03:46.876ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 393.688745ms iteration = 13 status = null
26082024-07-19T23:03:46.965ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
26092024-07-19T23:03:46.966ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "aee42164-caac-4172-82b4-504890e4c16b", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
26102024-07-19T23:03:46.966ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
26112024-07-19T23:03:46.967ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d24302f3-9e39-4cb6-b838-431995a4eb51", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
26122024-07-19T23:03:46.967ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
26132024-07-19T23:03:46.969ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
26142024-07-19T23:03:46.969ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
26152024-07-19T23:03:46.969ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 486.94016ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
26162024-07-19T23:03:47.054ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 348 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = b278a599-fd26-409a-b3d4-3327e7a8c0c8 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
26172024-07-19T23:03:47.057ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
26182024-07-19T23:03:47.057ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::c]:12221
26192024-07-19T23:03:47.067ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 584.421261ms iteration = 13 status = {}
26202024-07-19T23:03:47.069ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
26212024-07-19T23:03:47.069ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "53d0cea1-9bbe-4158-a5c0-a52bdffe921e", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26222024-07-19T23:03:47.069ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26232024-07-19T23:03:47.071ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "0d25a5f7-5646-49de-840b-4516a4288c9f", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26242024-07-19T23:03:47.071ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26252024-07-19T23:03:47.072ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "206c7319-7eda-4264-a1da-4fba705db1a9", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26262024-07-19T23:03:47.073ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26272024-07-19T23:03:47.074ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
26282024-07-19T23:03:47.074ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 592.342926ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
26292024-07-19T23:03:47.132ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/producers
26302024-07-19T23:03:47.132ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:775 local_addr = [fd00:1122:3344:101::c]:12221 remote_addr = [fd00:1122:3344:101::c]:44365
26312024-07-19T23:03:47.132ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44365 req_id = 5691a5e1-d095-426d-8fab-0fd848cc9e37 roles = RoleSet { roles: {} } uri = /metrics/producers
26322024-07-19T23:03:47.133ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44365 req_id = 5691a5e1-d095-426d-8fab-0fd848cc9e37 resource = Database result = Ok(()) uri = /metrics/producers
26332024-07-19T23:03:47.135ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26342024-07-19T23:03:47.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26352024-07-19T23:03:47.137ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
26362024-07-19T23:03:47.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26372024-07-19T23:03:47.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26382024-07-19T23:03:47.146ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
26392024-07-19T23:03:47.146ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7c5701ae-9f9a-4fb8-948a-ef4c3547c0cd", "content-length": "121", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
26402024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
26412024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
26422024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
26432024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
26442024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
26452024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
26462024-07-19T23:03:47.146ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
26472024-07-19T23:03:47.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26482024-07-19T23:03:47.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26492024-07-19T23:03:47.148ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
26502024-07-19T23:03:47.148ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7e9233d1-3ae1-4ba9-a655-32bae2f2071f", "content-length": "2", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
26512024-07-19T23:03:47.148ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
26522024-07-19T23:03:47.149ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "32857894-36cf-4386-971f-32cf40f1f9ae", "content-length": "52", "date": "Fri, 19 Jul 2024 23:03:46 GMT"} })
26532024-07-19T23:03:47.149ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
26542024-07-19T23:03:47.149ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26552024-07-19T23:03:47.169ZINFOnexus: registered oximeter collector client file = nexus/src/app/oximeter.rs:338 id = b9023b0a-a899-4caf-80bc-260e9a17ac56
26562024-07-19T23:03:47.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44365 req_id = 5691a5e1-d095-426d-8fab-0fd848cc9e37 roles = RoleSet { roles: {} } uri = /metrics/producers
26572024-07-19T23:03:47.169ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44365 req_id = 5691a5e1-d095-426d-8fab-0fd848cc9e37 resource = Database result = Ok(()) uri = /metrics/producers
26582024-07-19T23:03:47.199ZDEBGnexus: client request body = Some(Body) method = POST oximeter-collector = b9023b0a-a899-4caf-80bc-260e9a17ac56 uri = http://[fd00:1122:3344:101::d]:12223/producers
26592024-07-19T23:03:47.200ZDEBGnexus: client response oximeter-collector = b9023b0a-a899-4caf-80bc-260e9a17ac56 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::d)), port: Some(12223), path: "/producers", query: None, fragment: None }, status: 204, headers: {"x-request-id": "352b0ae8-60ab-4968-aad8-0b1b794816bd", "date": "Fri, 19 Jul 2024 23:03:47 GMT"} })
26602024-07-19T23:03:47.200ZINFOnexus: assigned collector to new producer collector_id = b9023b0a-a899-4caf-80bc-260e9a17ac56 file = nexus/src/app/oximeter.rs:125 producer_id = a88a8918-a4d7-44fb-8a6a-ae496052bdd5
26612024-07-19T23:03:47.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 69417 local_addr = [fd00:1122:3344:101::c]:12221 method = POST remote_addr = [fd00:1122:3344:101::c]:44365 req_id = 5691a5e1-d095-426d-8fab-0fd848cc9e37 response_code = 201 uri = /metrics/producers
26622024-07-19T23:03:47.202ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "f7970465-7ee5-4b6d-89e4-28505111454f", "date": "Fri, 19 Jul 2024 23:03:47 GMT"} })
26632024-07-19T23:03:47.202ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
26642024-07-19T23:03:47.202ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26652024-07-19T23:03:47.202ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::c)), port: Some(12221), path: "/metrics/producers", query: None, fragment: None }, status: 201, headers: {"content-type": "application/json", "x-request-id": "5691a5e1-d095-426d-8fab-0fd848cc9e37", "content-length": "41", "date": "Fri, 19 Jul 2024 23:03:47 GMT"} })
26662024-07-19T23:03:47.202ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f812e056-1df2-4142-b942-ead7e01ebcd1", "content-length": "494", "date": "Fri, 19 Jul 2024 23:03:47 GMT"} })
26672024-07-19T23:03:47.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26682024-07-19T23:03:47.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26692024-07-19T23:03:47.204ZDEBGnexus (producer-registration-task): registered with nexus successfully
26702024-07-19T23:03:47.204ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
26712024-07-19T23:03:47.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26722024-07-19T23:03:47.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26732024-07-19T23:03:47.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26742024-07-19T23:03:47.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26752024-07-19T23:03:47.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26762024-07-19T23:03:47.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26772024-07-19T23:03:47.216ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 732.695951ms iteration = 13 status = {}
26782024-07-19T23:03:56.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
26792024-07-19T23:03:56.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26802024-07-19T23:03:56.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26812024-07-19T23:03:56.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
26822024-07-19T23:03:56.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
26832024-07-19T23:03:56.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26842024-07-19T23:03:56.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26852024-07-19T23:03:56.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26862024-07-19T23:03:56.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26872024-07-19T23:03:56.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26882024-07-19T23:03:56.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
26892024-07-19T23:03:56.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.581105ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26902024-07-19T23:03:56.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26912024-07-19T23:03:56.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26922024-07-19T23:03:56.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26932024-07-19T23:03:56.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26942024-07-19T23:03:56.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26952024-07-19T23:03:56.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26962024-07-19T23:03:56.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
26972024-07-19T23:03:56.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26982024-07-19T23:03:56.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26992024-07-19T23:03:56.501ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
27002024-07-19T23:03:56.501ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.21114ms iteration = 38 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
27012024-07-19T23:03:57.202ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 423 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = aeec7ac3-de4e-44e9-a831-7be64daf97d8 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
27022024-07-19T23:04:06.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
27032024-07-19T23:04:06.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27042024-07-19T23:04:06.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27052024-07-19T23:04:06.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
27062024-07-19T23:04:06.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
27072024-07-19T23:04:06.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27082024-07-19T23:04:06.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27092024-07-19T23:04:06.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27102024-07-19T23:04:06.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27112024-07-19T23:04:06.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27122024-07-19T23:04:06.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
27132024-07-19T23:04:06.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.631287ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27142024-07-19T23:04:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27152024-07-19T23:04:06.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27162024-07-19T23:04:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27172024-07-19T23:04:06.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27182024-07-19T23:04:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27192024-07-19T23:04:06.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27202024-07-19T23:04:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
27212024-07-19T23:04:06.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27222024-07-19T23:04:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27232024-07-19T23:04:06.498ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
27242024-07-19T23:04:06.498ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.43128ms iteration = 39 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
27252024-07-19T23:04:07.202ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 648 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = e349bc56-d944-40d8-b92b-ffe62b938ffc response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
27262024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 14 reason = Timeout
27272024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
27282024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
27292024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
27302024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
27312024-07-19T23:04:16.482ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
27322024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
27332024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
27342024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27352024-07-19T23:04:16.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27362024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
27372024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
27382024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27392024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
27402024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
27412024-07-19T23:04:16.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
27422024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27432024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27442024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
27452024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
27462024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27472024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
27482024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27492024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
27502024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
27512024-07-19T23:04:16.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
27522024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27532024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27542024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27552024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27562024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27572024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27582024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27592024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27602024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27612024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
27622024-07-19T23:04:16.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
27632024-07-19T23:04:16.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27642024-07-19T23:04:16.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27652024-07-19T23:04:16.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27662024-07-19T23:04:16.485ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27672024-07-19T23:04:16.486ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
27682024-07-19T23:04:16.486ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.386302ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
27692024-07-19T23:04:16.486ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27702024-07-19T23:04:16.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27712024-07-19T23:04:16.486ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27722024-07-19T23:04:16.486ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2478, tv_nsec: 682909263 } })
27732024-07-19T23:04:16.487ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2478, tv_nsec: 683169688 } })
27742024-07-19T23:04:16.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
27752024-07-19T23:04:16.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.269318ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27762024-07-19T23:04:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
27772024-07-19T23:04:16.487ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
27782024-07-19T23:04:16.488ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27792024-07-19T23:04:16.488ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27802024-07-19T23:04:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
27812024-07-19T23:04:16.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27822024-07-19T23:04:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27832024-07-19T23:04:16.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27842024-07-19T23:04:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27852024-07-19T23:04:16.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27862024-07-19T23:04:16.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27872024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27882024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27892024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
27902024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27912024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27922024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27932024-07-19T23:04:16.490ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27942024-07-19T23:04:16.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27952024-07-19T23:04:16.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27962024-07-19T23:04:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27972024-07-19T23:04:16.492ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27982024-07-19T23:04:16.492ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2478, tv_nsec: 688470355 } })
27992024-07-19T23:04:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28002024-07-19T23:04:16.492ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
28012024-07-19T23:04:16.492ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28022024-07-19T23:04:16.492ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
28032024-07-19T23:04:16.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28042024-07-19T23:04:16.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28052024-07-19T23:04:16.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28062024-07-19T23:04:16.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28072024-07-19T23:04:16.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28082024-07-19T23:04:16.494ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28092024-07-19T23:04:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28102024-07-19T23:04:16.494ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28112024-07-19T23:04:16.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
28122024-07-19T23:04:16.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
28132024-07-19T23:04:16.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28142024-07-19T23:04:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28152024-07-19T23:04:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28162024-07-19T23:04:16.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28172024-07-19T23:04:16.497ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28182024-07-19T23:04:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28192024-07-19T23:04:16.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28202024-07-19T23:04:16.501ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
28212024-07-19T23:04:16.501ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 18.418797ms iteration = 15 status = {"physical_disks_added":0}
28222024-07-19T23:04:16.501ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
28232024-07-19T23:04:16.501ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.264794ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
28242024-07-19T23:04:16.503ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
28252024-07-19T23:04:16.503ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.473074ms iteration = 40 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
28262024-07-19T23:04:16.504ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28272024-07-19T23:04:16.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28282024-07-19T23:04:16.506ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28292024-07-19T23:04:16.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28302024-07-19T23:04:16.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28312024-07-19T23:04:16.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28322024-07-19T23:04:16.509ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28332024-07-19T23:04:16.510ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28342024-07-19T23:04:16.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28352024-07-19T23:04:16.511ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
28362024-07-19T23:04:16.511ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28372024-07-19T23:04:16.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28382024-07-19T23:04:16.511ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28392024-07-19T23:04:16.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28402024-07-19T23:04:16.512ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28412024-07-19T23:04:16.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
28422024-07-19T23:04:16.515ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28432024-07-19T23:04:16.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28442024-07-19T23:04:16.516ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.473302ms iteration = 14 status = 0
28452024-07-19T23:04:16.518ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28462024-07-19T23:04:16.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
28472024-07-19T23:04:16.520ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
28482024-07-19T23:04:16.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 37.269221ms iteration = 14 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
28492024-07-19T23:04:16.638ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28502024-07-19T23:04:16.639ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "06923739-f45f-4918-b1e0-0adc48d51bc9", "content-length": "372", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28512024-07-19T23:04:16.639ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28522024-07-19T23:04:16.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28532024-07-19T23:04:16.640ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28542024-07-19T23:04:16.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28552024-07-19T23:04:16.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28562024-07-19T23:04:16.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28572024-07-19T23:04:16.644ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
28582024-07-19T23:04:16.644ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "5db0fd1f-d7d4-4b42-a727-1441a8c6a0c0", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28592024-07-19T23:04:16.647ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 163.961657ms iteration = 14 status = {}
28602024-07-19T23:04:16.652ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
28612024-07-19T23:04:16.653ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "bb66eae0-f5c2-429c-818d-7be0a08a1292", "content-length": "691", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28622024-07-19T23:04:16.653ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
28632024-07-19T23:04:16.653ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
28642024-07-19T23:04:16.653ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
28652024-07-19T23:04:16.653ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 170.73024ms iteration = 14 status = {}
28662024-07-19T23:04:16.798ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
28672024-07-19T23:04:16.798ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28682024-07-19T23:04:16.798ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
28692024-07-19T23:04:16.798ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28702024-07-19T23:04:16.798ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
28712024-07-19T23:04:16.798ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28722024-07-19T23:04:16.798ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a2197492-b411-46ba-8325-38d7f0ce7cbd", "content-length": "26", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28732024-07-19T23:04:16.798ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28742024-07-19T23:04:16.798ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "07eba747-fd28-415c-8bd9-4fbe4e69d8de", "content-length": "26", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28752024-07-19T23:04:16.798ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28762024-07-19T23:04:16.798ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "437af2f5-7ad6-430e-a631-c9b616c6a806", "content-length": "26", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28772024-07-19T23:04:16.798ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28782024-07-19T23:04:16.801ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28792024-07-19T23:04:16.802ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
28802024-07-19T23:04:16.802ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28812024-07-19T23:04:16.885ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
28822024-07-19T23:04:16.886ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "a4b121f9-3525-45c1-85b3-2c988cf70608", "content-length": "2", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28832024-07-19T23:04:16.888ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
28842024-07-19T23:04:16.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
28852024-07-19T23:04:16.890ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 407.494778ms iteration = 14 status = null
28862024-07-19T23:04:16.974ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 491.862439ms iteration = 14 status = {}
28872024-07-19T23:04:17.026ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28882024-07-19T23:04:17.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28892024-07-19T23:04:17.029ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
28902024-07-19T23:04:17.029ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28912024-07-19T23:04:17.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28922024-07-19T23:04:17.037ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
28932024-07-19T23:04:17.038ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "3db2dcfc-47ca-4da7-9e84-a9449c955052", "content-length": "121", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
28942024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
28952024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
28962024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
28972024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
28982024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
28992024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
29002024-07-19T23:04:17.038ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
29012024-07-19T23:04:17.038ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29022024-07-19T23:04:17.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29032024-07-19T23:04:17.040ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
29042024-07-19T23:04:17.040ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4e052173-e4e3-4637-9eb0-c63e21014162", "content-length": "2", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
29052024-07-19T23:04:17.040ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
29062024-07-19T23:04:17.041ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f1795de3-15e0-4ab4-97fe-79d73a6dcba3", "content-length": "52", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
29072024-07-19T23:04:17.041ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
29082024-07-19T23:04:17.041ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
29092024-07-19T23:04:17.089ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "a6233c94-3379-42b7-bcf4-ed37befe6ddc", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
29102024-07-19T23:04:17.089ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
29112024-07-19T23:04:17.089ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
29122024-07-19T23:04:17.089ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "133c2658-6d61-45ef-860b-383272a15ac5", "content-length": "494", "date": "Fri, 19 Jul 2024 23:04:16 GMT"} })
29132024-07-19T23:04:17.089ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29142024-07-19T23:04:17.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29152024-07-19T23:04:17.093ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29162024-07-19T23:04:17.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29172024-07-19T23:04:17.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29182024-07-19T23:04:17.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29192024-07-19T23:04:17.097ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29202024-07-19T23:04:17.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29212024-07-19T23:04:17.104ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 621.221414ms iteration = 14 status = {}
29222024-07-19T23:04:17.201ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 378 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = edcfcd5a-30aa-41b8-8424-b4dafa2cbe23 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
29232024-07-19T23:04:26.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
29242024-07-19T23:04:26.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29252024-07-19T23:04:26.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29262024-07-19T23:04:26.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
29272024-07-19T23:04:26.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
29282024-07-19T23:04:26.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29292024-07-19T23:04:26.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29302024-07-19T23:04:26.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29312024-07-19T23:04:26.485ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29322024-07-19T23:04:26.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29332024-07-19T23:04:26.487ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
29342024-07-19T23:04:26.487ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.670847ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29352024-07-19T23:04:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29362024-07-19T23:04:26.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29372024-07-19T23:04:26.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29382024-07-19T23:04:26.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29392024-07-19T23:04:26.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29402024-07-19T23:04:26.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29412024-07-19T23:04:26.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
29422024-07-19T23:04:26.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29432024-07-19T23:04:26.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29442024-07-19T23:04:26.499ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
29452024-07-19T23:04:26.499ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.202494ms iteration = 41 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
29462024-07-19T23:04:27.202ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 585 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = dfe3608c-c87d-4006-a6f7-1f374a6b4ac1 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
29472024-07-19T23:04:36.482ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
29482024-07-19T23:04:36.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29492024-07-19T23:04:36.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29502024-07-19T23:04:36.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
29512024-07-19T23:04:36.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
29522024-07-19T23:04:36.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29532024-07-19T23:04:36.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29542024-07-19T23:04:36.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29552024-07-19T23:04:36.484ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
29562024-07-19T23:04:36.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
29572024-07-19T23:04:36.486ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
29582024-07-19T23:04:36.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.226119ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29592024-07-19T23:04:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
29602024-07-19T23:04:36.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29612024-07-19T23:04:36.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29622024-07-19T23:04:36.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29632024-07-19T23:04:36.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29642024-07-19T23:04:36.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29652024-07-19T23:04:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
29662024-07-19T23:04:36.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29672024-07-19T23:04:36.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29682024-07-19T23:04:36.508ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
29692024-07-19T23:04:36.508ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.576446ms iteration = 42 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
29702024-07-19T23:04:37.202ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 1478 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 6173112a-9e84-44b7-8a06-ecc4b35a1042 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
29712024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
29722024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
29732024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29742024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
29752024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
29762024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29772024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
29782024-07-19T23:04:46.483ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-07-19 22:54:46.483065042 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
29792024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
29802024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
29812024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29822024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
29832024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 15 reason = Timeout
29842024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29852024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29862024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
29872024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
29882024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29892024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
29902024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
29912024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29922024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
29932024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
29942024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
29952024-07-19T23:04:46.483ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:69 reason = blueprint disabled target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3
29962024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 16.371\u{b5}s iteration = 9 status = {"error":"blueprint disabled","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3"}
29972024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
29982024-07-19T23:04:46.483ZWARNnexus (BackgroundTasks): phantom disk task started background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:46
29992024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
30002024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
30012024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
30022024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
30032024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30042024-07-19T23:04:46.483ZINFOnexus (BackgroundTasks): lookup region port task started background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:56
30052024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30062024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30072024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
30082024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
30092024-07-19T23:04:46.483ZWARNnexus (BackgroundTasks): region replacement task started background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:64
30102024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30112024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
30122024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30132024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
30142024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30152024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
30162024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
30172024-07-19T23:04:46.483ZINFOnexus (BackgroundTasks): Abandoned VMM reaper running background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:76
30182024-07-19T23:04:46.483ZINFOnexus (BackgroundTasks): region replacement driver task started background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:231
30192024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30202024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
30212024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30222024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30232024-07-19T23:04:46.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
30242024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
30252024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30262024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30272024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30282024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30292024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30302024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30312024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30322024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
30332024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
30342024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30352024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
30362024-07-19T23:04:46.484ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0f5ab007-5c53-4a1d-880a-afbba1f1b219.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("5b151b39-a31a-4605-b3d7-169782f4e37d.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 2508, tv_nsec: 680454508 } })
30372024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30382024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
30392024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30402024-07-19T23:04:46.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30412024-07-19T23:04:46.485ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d7897bb-e108-4529-b11c-7e55a9a521fc.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4ec1dd64-fc59-46ad-8f12-dcafd769aba1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("74563217-f703-4c90-990d-5143d6aa0282.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 2508, tv_nsec: 681518878 } })
30422024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30432024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
30442024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.125937ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
30452024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30462024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30472024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30482024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30492024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30502024-07-19T23:04:46.486ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
30512024-07-19T23:04:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 3.750238ms iteration = 8 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
30522024-07-19T23:04:46.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30532024-07-19T23:04:46.487ZWARNnexus (BackgroundTasks): phantom disk task done background_task = phantom_disks file = nexus/src/app/background/tasks/phantom_disks.rs:93
30542024-07-19T23:04:46.487ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.223267ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
30552024-07-19T23:04:46.488ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30562024-07-19T23:04:46.488ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30572024-07-19T23:04:46.488ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-07-19 22:54:46.483065042 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
30582024-07-19T23:04:46.488ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.722985ms iteration = 8 status = {"expiration":"2024-07-19T22:54:46.483065042Z","pruned":[]}
30592024-07-19T23:04:46.490ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2508, tv_nsec: 686521269 } })
30602024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30612024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30622024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30632024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30642024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30652024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30662024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30672024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30682024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30692024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30702024-07-19T23:04:46.491ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30712024-07-19T23:04:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30722024-07-19T23:04:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30732024-07-19T23:04:46.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30742024-07-19T23:04:46.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30752024-07-19T23:04:46.493ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1033
30762024-07-19T23:04:46.493ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
30772024-07-19T23:04:46.493ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30782024-07-19T23:04:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30792024-07-19T23:04:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30802024-07-19T23:04:46.493ZWARNnexus (BackgroundTasks): physical disk adoption task started background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:100
30812024-07-19T23:04:46.493ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30822024-07-19T23:04:46.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30832024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30842024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30852024-07-19T23:04:46.494ZINFOnexus (BackgroundTasks): lookup region port task done background_task = lookup_region_port file = nexus/src/app/background/tasks/lookup_region_port.rs:150
30862024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.277717ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
30872024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30882024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30892024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
30902024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
30912024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
30922024-07-19T23:04:46.494ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30932024-07-19T23:04:46.495ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2508, tv_nsec: 691275576 } })
30942024-07-19T23:04:46.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30952024-07-19T23:04:46.495ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30962024-07-19T23:04:46.495ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
30972024-07-19T23:04:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
30982024-07-19T23:04:46.496ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30992024-07-19T23:04:46.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31002024-07-19T23:04:46.496ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31012024-07-19T23:04:46.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31022024-07-19T23:04:46.496ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31032024-07-19T23:04:46.497ZWARNnexus (BackgroundTasks): region replacement task done background_task = region_replacement file = nexus/src/app/background/tasks/region_replacement.rs:185
31042024-07-19T23:04:46.497ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.121358ms iteration = 15 status = {"region_replacement_started_err":0,"region_replacement_started_ok":0}
31052024-07-19T23:04:46.497ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31062024-07-19T23:04:46.498ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1033
31072024-07-19T23:04:46.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31082024-07-19T23:04:46.499ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31092024-07-19T23:04:46.499ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31102024-07-19T23:04:46.499ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:458 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0
31112024-07-19T23:04:46.499ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 15.86045ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0}
31122024-07-19T23:04:46.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31132024-07-19T23:04:46.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
31142024-07-19T23:04:46.500ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31152024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31162024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31172024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31182024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31192024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31202024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31212024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31222024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31232024-07-19T23:04:46.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31242024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
31252024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31262024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31272024-07-19T23:04:46.502ZWARNnexus (BackgroundTasks): physical disk adoption task done background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:174
31282024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.256092ms iteration = 16 status = {"physical_disks_added":0}
31292024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31302024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31312024-07-19T23:04:46.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31322024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31332024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 112437b7-60f1-4673-858f-9f7fcebc50c3, lookup_type: ById(112437b7-60f1-4673-858f-9f7fcebc50c3) } result = Ok(())
31342024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31352024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31362024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
31372024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
31382024-07-19T23:04:46.503ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31392024-07-19T23:04:46.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31402024-07-19T23:04:46.504ZINFOnexus (BackgroundTasks): region replacement driver task done background_task = region_replacement_driver file = nexus/src/app/background/tasks/region_replacement_driver.rs:238
31412024-07-19T23:04:46.504ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 20.843311ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31422024-07-19T23:04:46.504ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31432024-07-19T23:04:46.505ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31442024-07-19T23:04:46.506ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 2508, tv_nsec: 702380559 } })
31452024-07-19T23:04:46.506ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1033 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
31462024-07-19T23:04:46.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31472024-07-19T23:04:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31482024-07-19T23:04:46.509ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31492024-07-19T23:04:46.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31502024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31512024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31522024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
31532024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
31542024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
31552024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31562024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31572024-07-19T23:04:46.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31582024-07-19T23:04:46.511ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31592024-07-19T23:04:46.511ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31602024-07-19T23:04:46.512ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 64dc36b8-8e00-4b96-9d33-bc2bc5b798af dns_zone_name = control-plane.oxide.internal found_so_far = 49 version = 1
31612024-07-19T23:04:46.512ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
31622024-07-19T23:04:46.512ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-07-19 22:57:48.003217 UTC dns_group = internal generation = 1
31632024-07-19T23:04:46.512ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 29.182233ms iteration = 8 status = {"generation":1}
31642024-07-19T23:04:46.512ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31652024-07-19T23:04:46.512ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31662024-07-19T23:04:46.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31672024-07-19T23:04:46.514ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31682024-07-19T23:04:46.514ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31692024-07-19T23:04:46.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31702024-07-19T23:04:46.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31712024-07-19T23:04:46.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31722024-07-19T23:04:46.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31732024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31742024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31752024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31762024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 0 version = 2
31772024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31782024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31792024-07-19T23:04:46.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31802024-07-19T23:04:46.517ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31812024-07-19T23:04:46.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31822024-07-19T23:04:46.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31832024-07-19T23:04:46.518ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31842024-07-19T23:04:46.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31852024-07-19T23:04:46.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31862024-07-19T23:04:46.521ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31872024-07-19T23:04:46.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31882024-07-19T23:04:46.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31892024-07-19T23:04:46.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31902024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba, lookup_type: ById(48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba) } result = Ok(())
31912024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
31922024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
31932024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = a7981fb0-3e44-4dc4-a305-22e07814373f dns_zone_name = oxide.test found_so_far = 1 version = 2
31942024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31952024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-07-19 22:57:48.722526 UTC dns_group = external generation = 2
31962024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 44.695707ms iteration = 8 status = {"generation":2}
31972024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-07-19T22:57:46.302605Z, time_modified: 2024-07-19T22:57:46.302605Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, db_silo: Silo { identity: SiloIdentity { id: cc81324c-39dc-48ea-ade9-f1c9b649dc89, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-07-19T22:57:48.684560Z, time_modified: 2024-07-19T22:57:48.684560Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8" }] }) }
31982024-07-19T23:04:46.527ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 44.771529ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]}},"default_endpoint":{"silo_id":"cc81324c-39dc-48ea-ade9-f1c9b649dc89","tls_certs":["888af6e5f7e13376a0e815a8a9d93a94f60ec8ebaad5493a2e83d4f73e5d70d8"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
31992024-07-19T23:04:46.530ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 47.180422ms iteration = 15 status = 0
32002024-07-19T23:04:46.543ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0
32012024-07-19T23:04:46.543ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:189 found = 0 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0
32022024-07-19T23:04:46.543ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 59.64744ms iteration = 8 status = {"error":null,"error_count":0,"found":0,"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0}
32032024-07-19T23:04:46.661ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
32042024-07-19T23:04:46.701ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
32052024-07-19T23:04:46.701ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32062024-07-19T23:04:46.702ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "f50ad29c-ffa9-4a4e-abb4-c8893ab13e9d", "content-length": "26", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32072024-07-19T23:04:46.702ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32082024-07-19T23:04:46.704ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32092024-07-19T23:04:46.709ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32102024-07-19T23:04:46.710ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "9b4d45bc-b2d8-4438-bc5f-f5ba68f3a0b1", "content-length": "372", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32112024-07-19T23:04:46.710ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32122024-07-19T23:04:46.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32132024-07-19T23:04:46.711ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32142024-07-19T23:04:46.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32152024-07-19T23:04:46.713ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32162024-07-19T23:04:46.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32172024-07-19T23:04:46.716ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 original_time_created = 2024-07-19 22:57:34.689399 UTC target_id = 112437b7-60f1-4673-858f-9f7fcebc50c3 time_created = 2024-07-19 22:57:34.689399 UTC
32182024-07-19T23:04:46.716ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 233.119331ms iteration = 43 status = {"status":"target blueprint unchanged","target_id":"112437b7-60f1-4673-858f-9f7fcebc50c3","time_created":"2024-07-19T22:57:34.689399Z"}
32192024-07-19T23:04:46.716ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32202024-07-19T23:04:46.716ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = vpc_route_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/vpc-routes", query: None, fragment: None }, status: 204, headers: {"x-request-id": "889756b4-ef4f-40fc-9850-c73c77adfb91", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32212024-07-19T23:04:46.718ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1041 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 zone_address = fd00:1122:3344:101::2
32222024-07-19T23:04:46.718ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32232024-07-19T23:04:46.718ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "cae58be0-98b8-4358-bb10-4a6fe303cf8b", "content-length": "26", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32242024-07-19T23:04:46.718ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1046 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32252024-07-19T23:04:46.719ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 235.659827ms iteration = 15 status = {}
32262024-07-19T23:04:46.719ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
32272024-07-19T23:04:46.843ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
32282024-07-19T23:04:46.844ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::8)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "ceb9cadf-176b-44fe-b2f3-36ae4a34f794", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32292024-07-19T23:04:46.844ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32302024-07-19T23:04:46.845ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::9)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "4c2a1064-d22e-4034-89b9-0ab2abb33257", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32312024-07-19T23:04:46.845ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32322024-07-19T23:04:46.847ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32332024-07-19T23:04:46.847ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 363.86914ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
32342024-07-19T23:04:46.868ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
32352024-07-19T23:04:46.876ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
32362024-07-19T23:04:46.876ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/rpw/nat/ipv4/gen", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "673a5c86-415b-4971-b14c-65b60cbb096e", "content-length": "2", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32372024-07-19T23:04:46.877ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32382024-07-19T23:04:46.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32392024-07-19T23:04:46.878ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 395.572909ms iteration = 15 status = null
32402024-07-19T23:04:46.890ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1041 zone_address = fd00:1122:3344:101::2
32412024-07-19T23:04:46.890ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32422024-07-19T23:04:46.891ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12225), path: "/local/switch-id", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "237710aa-d6e6-4281-a5db-3c3731c1dc71", "content-length": "26", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32432024-07-19T23:04:46.891ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1046 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32442024-07-19T23:04:46.893ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1078 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32452024-07-19T23:04:46.935ZDEBGnexus (BackgroundTasks): client request SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32462024-07-19T23:04:46.936ZDEBGnexus (BackgroundTasks): client response SledAgent = 48cd94f5-f87f-4753-bbbe-9dbe9c9d5fba background_task = v2p_manager result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/v2p", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "7f08a635-4e6a-4679-b679-378f43cc21e2", "content-length": "691", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32472024-07-19T23:04:46.936ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
32482024-07-19T23:04:46.936ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
32492024-07-19T23:04:46.936ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
32502024-07-19T23:04:46.937ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 454.295613ms iteration = 15 status = {}
32512024-07-19T23:04:47.008ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
32522024-07-19T23:04:47.087ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 604.62637ms iteration = 15 status = {}
32532024-07-19T23:04:47.100ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
32542024-07-19T23:04:47.101ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:2::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "9215f868-0735-4ef1-8e8e-1cc8b8398e89", "date": "Fri, 19 Jul 2024 23:04:47 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32552024-07-19T23:04:47.101ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32562024-07-19T23:04:47.104ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:3::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "382e89d9-74d0-4277-8d59-86254ad8fd44", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32572024-07-19T23:04:47.104ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32582024-07-19T23:04:47.105ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:1::1)), port: Some(5353), path: "/config", query: None, fragment: None }, status: 204, headers: {"x-request-id": "32901a3a-2fb9-4926-b587-26d8b2f37493", "date": "Fri, 19 Jul 2024 23:04:47 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32592024-07-19T23:04:47.106ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32602024-07-19T23:04:47.107ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
32612024-07-19T23:04:47.107ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 624.549595ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
32622024-07-19T23:04:47.125ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32632024-07-19T23:04:47.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32642024-07-19T23:04:47.127ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:146 port = SwitchPort { id: 7480b5d7-113f-4f2c-8ae6-adea12b95e42, rack_id: 858ba1b7-0444-4ae8-8154-caf799e42fa8, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6) } rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
32652024-07-19T23:04:47.127ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32662024-07-19T23:04:47.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32672024-07-19T23:04:47.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
32682024-07-19T23:04:47.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/port/qsfp0/settings", query: Some("tag=omicron"), fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "4bcad846-3175-4a44-b06d-6c4cb7b9430a", "content-length": "121", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32692024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1663 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: None, kr: false, lane: Some(LinkId(0)), speed: Speed40G } }} } switch_port_id = "qsfp0"
32702024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1690 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 90a14fdc-4c47-49d7-b5e6-acfdb6a54cf6
32712024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
32722024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None)}}
32732024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
32742024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 routes = {}
32752024-07-19T23:04:47.133ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8
32762024-07-19T23:04:47.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32772024-07-19T23:04:47.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32782024-07-19T23:04:47.135ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
32792024-07-19T23:04:47.135ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv4", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "585af654-602b-42ab-ad85-9d8de1cd876a", "content-length": "2", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32802024-07-19T23:04:47.135ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
32812024-07-19T23:04:47.135ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::2)), port: Some(12224), path: "/loopback/ipv6", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "88327741-2e1f-4a54-89d2-75e6cbbc4120", "content-length": "52", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32822024-07-19T23:04:47.135ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], port: "qsfp0" }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch_location = Switch0
32832024-07-19T23:04:47.135ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32842024-07-19T23:04:47.201ZINFOnexus (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/7b594d0/dropshot/src/server.rs:914 latency_us = 327 local_addr = [fd00:1122:3344:101::c]:39928 method = GET remote_addr = [fd00:1122:3344:101::d]:60126 req_id = 0b1a0b81-efb9-4582-aa01-476306dda945 response_code = 200 uri = /a88a8918-a4d7-44fb-8a6a-ae496052bdd5
32852024-07-19T23:04:47.233ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/switch-ports", query: None, fragment: None }, status: 204, headers: {"x-request-id": "d1b134bd-8002-4a39-8320-a123dc021be7", "date": "Fri, 19 Jul 2024 23:04:46 GMT"} })
32862024-07-19T23:04:47.233ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 switch = Switch0
32872024-07-19T23:04:47.233ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
32882024-07-19T23:04:47.233ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 858ba1b7-0444-4ae8-8154-caf799e42fa8 result = Ok(Response { url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv6(fd00:1122:3344:101::1)), port: Some(12345), path: "/network-bootstore-config", query: None, fragment: None }, status: 200, headers: {"content-type": "application/json", "x-request-id": "d809fe89-93ae-44c7-91d4-99db55f4147d", "content-length": "494", "date": "Fri, 19 Jul 2024 23:04:47 GMT"} })
32892024-07-19T23:04:47.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32902024-07-19T23:04:47.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32912024-07-19T23:04:47.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32922024-07-19T23:04:47.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32932024-07-19T23:04:47.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32942024-07-19T23:04:47.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32952024-07-19T23:04:47.240ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32962024-07-19T23:04:47.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32972024-07-19T23:04:47.247ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 764.061544ms iteration = 15 status = {}