Max level shown:
12025-03-25T01:30:01.333ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 106663aa-58b7-467d-b5d7-a0b57f3e9701 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22025-03-25T01:30:01.334ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 106663aa-58b7-467d-b5d7-a0b57f3e9701 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
32025-03-25T01:30:01.336ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 106663aa-58b7-467d-b5d7-a0b57f3e9701 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
42025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
52025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
62025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
82025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
92025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
102025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
112025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
122025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
142025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
152025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
162025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
172025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
182025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 22 reason = Timeout
192025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
202025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
212025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
222025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
232025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
242025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
252025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
262025-03-25T01:30:01.740ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
272025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
282025-03-25T01:30:01.740ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
292025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
302025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
312025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
322025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
332025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
342025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
352025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
362025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
372025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
382025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
392025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
402025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
412025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
422025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
432025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
442025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
452025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
462025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
472025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
482025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
492025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
502025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
512025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
522025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
532025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
542025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
552025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
562025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
572025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
582025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
592025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
602025-03-25T01:30:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
612025-03-25T01:30:01.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
622025-03-25T01:30:01.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
632025-03-25T01:30:01.745ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
642025-03-25T01:30:01.745ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.938721ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
652025-03-25T01:30:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
662025-03-25T01:30:01.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
672025-03-25T01:30:01.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
682025-03-25T01:30:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
692025-03-25T01:30:01.749ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
702025-03-25T01:30:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
712025-03-25T01:30:01.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
722025-03-25T01:30:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
732025-03-25T01:30:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
742025-03-25T01:30:01.751ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
752025-03-25T01:30:01.753ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1505, tv_nsec: 463845695 } })
762025-03-25T01:30:01.753ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
772025-03-25T01:30:01.754ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
782025-03-25T01:30:01.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
792025-03-25T01:30:01.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
802025-03-25T01:30:01.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
812025-03-25T01:30:01.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
822025-03-25T01:30:01.758ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
832025-03-25T01:30:01.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
842025-03-25T01:30:01.763ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1505, tv_nsec: 473616605 } })
852025-03-25T01:30:01.763ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
862025-03-25T01:30:01.768ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1505, tv_nsec: 479135753 } })
872025-03-25T01:30:01.769ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
882025-03-25T01:30:01.784ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
892025-03-25T01:30:01.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
902025-03-25T01:30:01.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
912025-03-25T01:30:01.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
922025-03-25T01:30:01.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
932025-03-25T01:30:01.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
942025-03-25T01:30:01.824ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 83.676937ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
952025-03-25T01:30:01.824ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
962025-03-25T01:30:01.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
972025-03-25T01:30:01.827ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 86.918863ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
982025-03-25T01:30:01.829ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 89.019744ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
992025-03-25T01:30:01.832ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1002025-03-25T01:30:01.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
1012025-03-25T01:30:01.835ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1022025-03-25T01:30:01.836ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1032025-03-25T01:30:01.836ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1042025-03-25T01:30:01.836ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1052025-03-25T01:30:01.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1062025-03-25T01:30:01.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1072025-03-25T01:30:01.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1082025-03-25T01:30:01.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1092025-03-25T01:30:01.840ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1102025-03-25T01:30:01.840ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1112025-03-25T01:30:01.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1122025-03-25T01:30:01.840ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1132025-03-25T01:30:01.840ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1142025-03-25T01:30:01.840ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1152025-03-25T01:30:01.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1162025-03-25T01:30:01.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1172025-03-25T01:30:01.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1182025-03-25T01:30:01.844ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1192025-03-25T01:30:01.844ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1202025-03-25T01:30:01.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1212025-03-25T01:30:01.846ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1222025-03-25T01:30:01.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1232025-03-25T01:30:01.847ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1242025-03-25T01:30:01.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1252025-03-25T01:30:01.850ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1262025-03-25T01:30:01.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1272025-03-25T01:30:01.851ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1282025-03-25T01:30:01.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1292025-03-25T01:30:01.862ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1302025-03-25T01:30:01.867ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
1312025-03-25T01:30:01.867ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1322025-03-25T01:30:01.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1332025-03-25T01:30:01.870ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1342025-03-25T01:30:01.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
1352025-03-25T01:30:01.874ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1362025-03-25T01:30:01.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
1372025-03-25T01:30:01.879ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1382025-03-25T01:30:01.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1392025-03-25T01:30:01.882ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1402025-03-25T01:30:01.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1412025-03-25T01:30:01.885ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
1422025-03-25T01:30:01.885ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 144.090272ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
1432025-03-25T01:30:01.888ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 147.199286ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1442025-03-25T01:30:01.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1452025-03-25T01:30:01.892ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
1462025-03-25T01:30:01.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1472025-03-25T01:30:01.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1482025-03-25T01:30:01.897ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 156.877685ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1492025-03-25T01:30:01.901ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
1502025-03-25T01:30:01.901ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1512025-03-25T01:30:01.901ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "05ef7cbf-cccf-4ea9-883e-4c2283f0b4cf", "content-length": "26", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
1522025-03-25T01:30:01.901ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1532025-03-25T01:30:01.903ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1542025-03-25T01:30:01.908ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
1552025-03-25T01:30:01.908ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 168.160646ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
1562025-03-25T01:30:01.915ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1572025-03-25T01:30:01.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1582025-03-25T01:30:01.920ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1592025-03-25T01:30:01.920ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1602025-03-25T01:30:01.920ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 179.138913ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1612025-03-25T01:30:01.921ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 180.807608ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1622025-03-25T01:30:01.927ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
1632025-03-25T01:30:01.927ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1642025-03-25T01:30:01.927ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 186.851093ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
1652025-03-25T01:30:01.931ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 1768bc4c-c9ec-4fc7-a560-5ef338d49e99, lookup_type: ById(1768bc4c-c9ec-4fc7-a560-5ef338d49e99) } result = Ok(())
1662025-03-25T01:30:01.931ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1672025-03-25T01:30:01.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1682025-03-25T01:30:01.932ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd zone_address = fd00:1122:3344:101::2
1692025-03-25T01:30:01.932ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1702025-03-25T01:30:01.933ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "26e99179-453d-4631-8df4-ffa3a141eac1", "content-length": "26", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
1712025-03-25T01:30:01.933ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1722025-03-25T01:30:01.934ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 193.208465ms iteration = 22 status = 0
1732025-03-25T01:30:01.935ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
1742025-03-25T01:30:01.958ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
1752025-03-25T01:30:01.958ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1762025-03-25T01:30:01.959ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d94f58b-b3b2-4fe7-8fe1-af57b0e64b85", "content-length": "26", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
1772025-03-25T01:30:01.959ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1782025-03-25T01:30:01.960ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1792025-03-25T01:30:01.970ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 229.622885ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
1802025-03-25T01:30:02.020ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 279.264676ms iteration = 22 status = {}
1812025-03-25T01:30:02.032ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1822025-03-25T01:30:02.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1832025-03-25T01:30:02.041ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1842025-03-25T01:30:02.045ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1852025-03-25T01:30:02.045ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1862025-03-25T01:30:02.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1872025-03-25T01:30:02.048ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1882025-03-25T01:30:02.051ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1892025-03-25T01:30:02.051ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1902025-03-25T01:30:02.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1912025-03-25T01:30:02.057ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 316.371207ms iteration = 25 status = {"physical_disks_added":0}
1922025-03-25T01:30:02.073ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1932025-03-25T01:30:02.074ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f8d2141-77f3-49a7-8bc3-049857689c8c", "content-length": "1", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
1942025-03-25T01:30:02.082ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1952025-03-25T01:30:02.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1962025-03-25T01:30:02.105ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1972025-03-25T01:30:02.105ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1982025-03-25T01:30:02.105ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1ff56a8-3427-4fa7-b441-6eaecea5df4c", "content-length": "372", "date": "Tue, 25 Mar 2025 01:30:02 GMT"} })
1992025-03-25T01:30:02.105ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2002025-03-25T01:30:02.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2012025-03-25T01:30:02.108ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 367.536229ms iteration = 22 status = null
2022025-03-25T01:30:02.169ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
2032025-03-25T01:30:02.170ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "41668507-103e-4db4-930b-190a0c404839", "content-length": "691", "date": "Tue, 25 Mar 2025 01:30:02 GMT"} })
2042025-03-25T01:30:02.170ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
2052025-03-25T01:30:02.170ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
2062025-03-25T01:30:02.170ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
2072025-03-25T01:30:02.172ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 431.89423ms iteration = 22 status = {}
2082025-03-25T01:30:02.199ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2092025-03-25T01:30:02.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2102025-03-25T01:30:02.202ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1a5160a0-1d63-486b-827e-4b7de2ad2079, rack_id: 16a011db-af85-49cc-8fa6-b468f46a78bd, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(80c078c2-e0e7-4e11-a9de-db9b8488eb2c) } rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
2112025-03-25T01:30:02.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2122025-03-25T01:30:02.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2132025-03-25T01:30:02.217ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2142025-03-25T01:30:02.217ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "bebc2664-cafd-4385-9ab4-5b84c2c6089c", "content-length": "134", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
2152025-03-25T01:30:02.217ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2162025-03-25T01:30:02.217ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 80c078c2-e0e7-4e11-a9de-db9b8488eb2c
2172025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2182025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2192025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2202025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2212025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2222025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2232025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2242025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2252025-03-25T01:30:02.218ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
2262025-03-25T01:30:02.219ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2272025-03-25T01:30:02.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2282025-03-25T01:30:02.222ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2292025-03-25T01:30:02.222ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "cafcc0b6-4df5-4551-a926-d4bde2868009", "content-length": "2", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
2302025-03-25T01:30:02.222ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2312025-03-25T01:30:02.222ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "b85184be-d728-4c44-be92-f304fc82a94b", "content-length": "51", "date": "Tue, 25 Mar 2025 01:30:01 GMT"} })
2322025-03-25T01:30:02.222ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
2332025-03-25T01:30:02.222ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2342025-03-25T01:30:02.236ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
2352025-03-25T01:30:02.236ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
2362025-03-25T01:30:02.425ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "28669bb2-3c79-43c3-895e-f763c63229ef", "date": "Tue, 25 Mar 2025 01:30:02 GMT"} })
2372025-03-25T01:30:02.425ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch = Switch0
2382025-03-25T01:30:02.425ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2392025-03-25T01:30:02.425ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f10dde40-4cee-46c7-abcc-0f7d9d2fc2be roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2402025-03-25T01:30:02.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f10dde40-4cee-46c7-abcc-0f7d9d2fc2be resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2412025-03-25T01:30:02.426ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c64c9c1b-1023-4175-9333-6e9e8e4ae892", "content-length": "682", "date": "Tue, 25 Mar 2025 01:30:02 GMT"} })
2422025-03-25T01:30:02.426ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2432025-03-25T01:30:02.426ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3f0b19c0-11b4-47d7-b65c-b6ce1dfaee83", "date": "Tue, 25 Mar 2025 01:30:02 GMT"} })
2442025-03-25T01:30:02.426ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2452025-03-25T01:30:02.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2462025-03-25T01:30:02.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2472025-03-25T01:30:02.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2889 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f10dde40-4cee-46c7-abcc-0f7d9d2fc2be response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2482025-03-25T01:30:02.429ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2492025-03-25T01:30:02.430ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2502025-03-25T01:30:02.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2512025-03-25T01:30:02.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2522025-03-25T01:30:02.432ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2532025-03-25T01:30:02.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2542025-03-25T01:30:02.432ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2552025-03-25T01:30:02.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2562025-03-25T01:30:02.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2572025-03-25T01:30:02.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2582025-03-25T01:30:02.436ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2592025-03-25T01:30:02.436ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2602025-03-25T01:30:02.442ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 702.003005ms iteration = 22 status = {}
2612025-03-25T01:30:02.442ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "64b104d3-9343-4825-9ace-33a52841bf76", "date": "Tue, 25 Mar 2025 01:30:02 GMT"} })
2622025-03-25T01:30:02.445ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 704.246317ms iteration = 22 status = {}
2632025-03-25T01:30:03.460ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 87bd4e15-6349-4a6b-b9d7-7a282bc8994f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2642025-03-25T01:30:03.461ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 87bd4e15-6349-4a6b-b9d7-7a282bc8994f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2652025-03-25T01:30:03.462ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2098 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 87bd4e15-6349-4a6b-b9d7-7a282bc8994f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2662025-03-25T01:30:04.483ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dd575fc6-3747-40a7-9984-94a667dd756e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2672025-03-25T01:30:04.483ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dd575fc6-3747-40a7-9984-94a667dd756e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2682025-03-25T01:30:04.485ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2135 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dd575fc6-3747-40a7-9984-94a667dd756e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2692025-03-25T01:30:05.506ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6786d9cd-7cdd-410b-b3cd-a710045675cb roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2702025-03-25T01:30:05.506ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6786d9cd-7cdd-410b-b3cd-a710045675cb resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2712025-03-25T01:30:05.508ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2224 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6786d9cd-7cdd-410b-b3cd-a710045675cb response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2722025-03-25T01:30:05.767ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 622 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 4bf5a8f4-3173-40aa-bed6-80b2d58a76e2 response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
2732025-03-25T01:30:06.527ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c6745bcd-b8fc-4d8a-8574-2ba081dedfb9 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2742025-03-25T01:30:06.528ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c6745bcd-b8fc-4d8a-8574-2ba081dedfb9 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2752025-03-25T01:30:06.530ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2393 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c6745bcd-b8fc-4d8a-8574-2ba081dedfb9 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2762025-03-25T01:30:07.550ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = d67b6e44-d907-48f2-bc52-dc0c7e3b28f5 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2772025-03-25T01:30:07.550ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = d67b6e44-d907-48f2-bc52-dc0c7e3b28f5 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2782025-03-25T01:30:07.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1983 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = d67b6e44-d907-48f2-bc52-dc0c7e3b28f5 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2792025-03-25T01:30:08.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = fb0933a3-5c60-4033-829b-300c19c89159 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2802025-03-25T01:30:08.573ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = fb0933a3-5c60-4033-829b-300c19c89159 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2812025-03-25T01:30:08.575ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2458 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = fb0933a3-5c60-4033-829b-300c19c89159 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2822025-03-25T01:30:09.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 773793d2-8692-45fa-b6dd-a16d21496e75 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2832025-03-25T01:30:09.595ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 773793d2-8692-45fa-b6dd-a16d21496e75 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2842025-03-25T01:30:09.597ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2456 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 773793d2-8692-45fa-b6dd-a16d21496e75 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2852025-03-25T01:30:10.617ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 60df697e-b554-4af7-9fd0-2268ef1ec306 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2862025-03-25T01:30:10.618ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 60df697e-b554-4af7-9fd0-2268ef1ec306 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2872025-03-25T01:30:10.620ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2366 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 60df697e-b554-4af7-9fd0-2268ef1ec306 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2882025-03-25T01:30:11.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = da09b587-ee4d-447c-ac44-a7aef6a9b0ef roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2892025-03-25T01:30:11.641ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = da09b587-ee4d-447c-ac44-a7aef6a9b0ef resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2902025-03-25T01:30:11.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1731 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = da09b587-ee4d-447c-ac44-a7aef6a9b0ef response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
2912025-03-25T01:30:11.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
2922025-03-25T01:30:11.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2932025-03-25T01:30:11.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
2942025-03-25T01:30:11.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
2952025-03-25T01:30:11.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2962025-03-25T01:30:11.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2972025-03-25T01:30:11.770ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2982025-03-25T01:30:11.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2992025-03-25T01:30:11.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3002025-03-25T01:30:11.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3012025-03-25T01:30:11.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3022025-03-25T01:30:11.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3032025-03-25T01:30:11.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3042025-03-25T01:30:11.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3052025-03-25T01:30:11.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3062025-03-25T01:30:11.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
3072025-03-25T01:30:11.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3082025-03-25T01:30:11.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3092025-03-25T01:30:11.788ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
3102025-03-25T01:30:11.788ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.377728ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
3112025-03-25T01:30:11.801ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.579517ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3122025-03-25T01:30:12.662ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f33b0ad4-146e-4ce4-aacc-cc3ccb291a99 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3132025-03-25T01:30:12.663ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f33b0ad4-146e-4ce4-aacc-cc3ccb291a99 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3142025-03-25T01:30:12.664ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2099 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f33b0ad4-146e-4ce4-aacc-cc3ccb291a99 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3152025-03-25T01:30:13.685ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4d9c4b59-46ec-424d-86f3-2e9724370025 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3162025-03-25T01:30:13.685ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4d9c4b59-46ec-424d-86f3-2e9724370025 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3172025-03-25T01:30:13.687ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2298 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4d9c4b59-46ec-424d-86f3-2e9724370025 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3182025-03-25T01:30:14.707ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ce690e72-f57f-4dab-832f-955b40dfa03d roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3192025-03-25T01:30:14.707ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ce690e72-f57f-4dab-832f-955b40dfa03d resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3202025-03-25T01:30:14.709ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1736 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ce690e72-f57f-4dab-832f-955b40dfa03d response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3212025-03-25T01:30:15.729ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = aedd45db-df57-42fd-a65b-f50d66107414 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3222025-03-25T01:30:15.730ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = aedd45db-df57-42fd-a65b-f50d66107414 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3232025-03-25T01:30:15.732ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2371 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = aedd45db-df57-42fd-a65b-f50d66107414 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3242025-03-25T01:30:15.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62439 req_id = 874f4ef8-458c-4717-a740-70359ba750eb roles = RoleSet { roles: {} } uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?sort_by=id_ascending
3252025-03-25T01:30:15.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62439 req_id = 874f4ef8-458c-4717-a740-70359ba750eb resource = Database result = Ok(()) uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?sort_by=id_ascending
3262025-03-25T01:30:15.763ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3355 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62439 req_id = 874f4ef8-458c-4717-a740-70359ba750eb response_code = 200 uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?sort_by=id_ascending
3272025-03-25T01:30:15.763ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62439 req_id = b84cde7c-3b31-46be-85d9-493242be9045 roles = RoleSet { roles: {} } uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4MzI5NzcxOC01ZjQyLTQ3NmMtYTA2Ni1iODU2ZmRlODVmMjUifX0%3D
3282025-03-25T01:30:15.764ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62439 req_id = b84cde7c-3b31-46be-85d9-493242be9045 resource = Database result = Ok(()) uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4MzI5NzcxOC01ZjQyLTQ3NmMtYTA2Ni1iODU2ZmRlODVmMjUifX0%3D
3292025-03-25T01:30:15.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2144 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:62439 req_id = b84cde7c-3b31-46be-85d9-493242be9045 response_code = 200 uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4MzI5NzcxOC01ZjQyLTQ3NmMtYTA2Ni1iODU2ZmRlODVmMjUifX0%3D
3302025-03-25T01:30:15.767ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 408 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = ee0093fe-6758-44b8-9fb3-700f7ff42fa5 response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
3312025-03-25T01:30:16.753ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 077622d8-8c23-44ff-80c2-28a746d7d726 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3322025-03-25T01:30:16.753ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 077622d8-8c23-44ff-80c2-28a746d7d726 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3332025-03-25T01:30:16.755ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 077622d8-8c23-44ff-80c2-28a746d7d726 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3342025-03-25T01:30:17.775ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 391cbd55-259f-4dc0-a5c8-7bd06a6fc152 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3352025-03-25T01:30:17.775ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 391cbd55-259f-4dc0-a5c8-7bd06a6fc152 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3362025-03-25T01:30:17.776ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1818 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 391cbd55-259f-4dc0-a5c8-7bd06a6fc152 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3372025-03-25T01:30:18.795ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5cec0061-4da3-404e-9497-38edd57da453 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3382025-03-25T01:30:18.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5cec0061-4da3-404e-9497-38edd57da453 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3392025-03-25T01:30:18.797ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2030 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5cec0061-4da3-404e-9497-38edd57da453 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3402025-03-25T01:30:19.818ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e85b06b3-1cbc-4c99-9ff9-c3d9e3ea31b7 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3412025-03-25T01:30:19.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e85b06b3-1cbc-4c99-9ff9-c3d9e3ea31b7 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3422025-03-25T01:30:19.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2610 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e85b06b3-1cbc-4c99-9ff9-c3d9e3ea31b7 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3432025-03-25T01:30:20.842ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 03c16207-7e07-4268-9f4f-9d21c73e6403 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3442025-03-25T01:30:20.842ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 03c16207-7e07-4268-9f4f-9d21c73e6403 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3452025-03-25T01:30:20.844ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1961 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 03c16207-7e07-4268-9f4f-9d21c73e6403 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3462025-03-25T01:30:21.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
3472025-03-25T01:30:21.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3482025-03-25T01:30:21.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3492025-03-25T01:30:21.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
3502025-03-25T01:30:21.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3512025-03-25T01:30:21.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3522025-03-25T01:30:21.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3532025-03-25T01:30:21.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
3542025-03-25T01:30:21.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3552025-03-25T01:30:21.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3562025-03-25T01:30:21.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3572025-03-25T01:30:21.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3582025-03-25T01:30:21.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3592025-03-25T01:30:21.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
3602025-03-25T01:30:21.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3612025-03-25T01:30:21.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3622025-03-25T01:30:21.759ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
3632025-03-25T01:30:21.759ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.341426ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
3642025-03-25T01:30:21.772ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3652025-03-25T01:30:21.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3662025-03-25T01:30:21.774ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.743398ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
3672025-03-25T01:30:21.864ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f0dbd369-6ff2-43c7-857f-f93c4e84915a roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3682025-03-25T01:30:21.865ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f0dbd369-6ff2-43c7-857f-f93c4e84915a resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3692025-03-25T01:30:21.866ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1521 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f0dbd369-6ff2-43c7-857f-f93c4e84915a response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3702025-03-25T01:30:22.886ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5056c9e8-4004-4918-af77-00535b70e744 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3712025-03-25T01:30:22.886ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5056c9e8-4004-4918-af77-00535b70e744 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3722025-03-25T01:30:22.888ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2014 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5056c9e8-4004-4918-af77-00535b70e744 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3732025-03-25T01:30:23.908ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2a359dff-6881-454c-b411-38c5d9688125 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3742025-03-25T01:30:23.908ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2a359dff-6881-454c-b411-38c5d9688125 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3752025-03-25T01:30:23.910ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2176 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2a359dff-6881-454c-b411-38c5d9688125 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3762025-03-25T01:30:24.930ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c4ab3046-0fd8-42b6-94bd-4ba288726fd2 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3772025-03-25T01:30:24.930ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c4ab3046-0fd8-42b6-94bd-4ba288726fd2 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3782025-03-25T01:30:24.933ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3128 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c4ab3046-0fd8-42b6-94bd-4ba288726fd2 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3792025-03-25T01:30:25.767ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 771 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 9fe46fde-eaab-482e-a88c-7bc692ef0afc response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
3802025-03-25T01:30:25.957ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b6edd974-c24c-4217-9f4e-4bfa105ba391 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3812025-03-25T01:30:25.957ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b6edd974-c24c-4217-9f4e-4bfa105ba391 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3822025-03-25T01:30:25.958ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b6edd974-c24c-4217-9f4e-4bfa105ba391 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3832025-03-25T01:30:26.978ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 1df52d72-b56e-4d63-9e8a-4259b5d1ea8a roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3842025-03-25T01:30:26.978ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 1df52d72-b56e-4d63-9e8a-4259b5d1ea8a resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3852025-03-25T01:30:26.980ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 1df52d72-b56e-4d63-9e8a-4259b5d1ea8a response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3862025-03-25T01:30:28.001ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 743a62d2-195b-4b55-9eac-7b88c021e0eb roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3872025-03-25T01:30:28.001ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 743a62d2-195b-4b55-9eac-7b88c021e0eb resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3882025-03-25T01:30:28.003ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2041 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 743a62d2-195b-4b55-9eac-7b88c021e0eb response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3892025-03-25T01:30:29.022ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2814c639-3374-4f5c-930f-a49b9c586533 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3902025-03-25T01:30:29.022ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2814c639-3374-4f5c-930f-a49b9c586533 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3912025-03-25T01:30:29.024ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1973 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2814c639-3374-4f5c-930f-a49b9c586533 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3922025-03-25T01:30:30.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6fa12165-1eaa-47c8-b41e-bf762b152aa1 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3932025-03-25T01:30:30.046ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6fa12165-1eaa-47c8-b41e-bf762b152aa1 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3942025-03-25T01:30:30.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2557 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6fa12165-1eaa-47c8-b41e-bf762b152aa1 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3952025-03-25T01:30:31.068ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7ec4bf34-effe-4bde-a36d-0e03dc0da655 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3962025-03-25T01:30:31.068ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7ec4bf34-effe-4bde-a36d-0e03dc0da655 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3972025-03-25T01:30:31.070ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2115 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7ec4bf34-effe-4bde-a36d-0e03dc0da655 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
3982025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
3992025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
4002025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4012025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4022025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
4032025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
4042025-03-25T01:30:31.740ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
4052025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4062025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
4072025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 23 reason = Timeout
4082025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
4092025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
4102025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
4112025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4122025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
4132025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
4142025-03-25T01:30:31.740ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b
4152025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
4162025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.31\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b"}
4172025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
4182025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4192025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
4202025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
4212025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
4222025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4232025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4242025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
4252025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4262025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
4272025-03-25T01:30:31.740ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
4282025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4292025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
4302025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
4312025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
4322025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
4332025-03-25T01:30:31.741ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-25 01:20:31.740889237 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
4342025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
4352025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4362025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
4372025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
4382025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
4392025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4402025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
4412025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
4422025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
4432025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
4442025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
4452025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4462025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
4472025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
4482025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
4492025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
4502025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
4512025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4522025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
4532025-03-25T01:30:31.741ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4542025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
4552025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4562025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4572025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
4582025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4592025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4602025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
4612025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
4622025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
4632025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4642025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
4652025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
4662025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4672025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
4682025-03-25T01:30:31.741ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4b6e96f6-cd23-4897-9bfb-96b629ccf872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("658f8989-b24e-48cb-83e6-b3a3b15d47d4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("658f8989-b24e-48cb-83e6-b3a3b15d47d4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1535, tv_nsec: 451405167 } })
4692025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
4702025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4712025-03-25T01:30:31.741ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f0307c3-79bb-4dd8-87b6-ae6655144209.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40054b36-2618-459b-aad0-799de8938fb6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b6a22759-03b9-4dff-b9a0-d413b6908c97.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b6a22759-03b9-4dff-b9a0-d413b6908c97.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1535, tv_nsec: 451438577 } })
4722025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4732025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4742025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4752025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4762025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4772025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4782025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
4792025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4802025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
4812025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4822025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
4832025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4842025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
4852025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
4862025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4872025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4882025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4892025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4902025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4912025-03-25T01:30:31.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4922025-03-25T01:30:31.742ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
4932025-03-25T01:30:31.742ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
4942025-03-25T01:30:31.742ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.415881ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
4952025-03-25T01:30:31.742ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.313989ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
4962025-03-25T01:30:31.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4972025-03-25T01:30:31.744ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4982025-03-25T01:30:31.744ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4992025-03-25T01:30:31.745ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1535, tv_nsec: 455038689 } })
5002025-03-25T01:30:31.745ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
5012025-03-25T01:30:31.745ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.067783ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5022025-03-25T01:30:31.746ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5032025-03-25T01:30:31.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
5042025-03-25T01:30:31.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5052025-03-25T01:30:31.747ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
5062025-03-25T01:30:31.747ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5072025-03-25T01:30:31.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5082025-03-25T01:30:31.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5092025-03-25T01:30:31.748ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
5102025-03-25T01:30:31.748ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5112025-03-25T01:30:31.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5122025-03-25T01:30:31.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
5132025-03-25T01:30:31.749ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5142025-03-25T01:30:31.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5152025-03-25T01:30:31.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
5162025-03-25T01:30:31.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
5172025-03-25T01:30:31.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5182025-03-25T01:30:31.754ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5192025-03-25T01:30:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5202025-03-25T01:30:31.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5212025-03-25T01:30:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5222025-03-25T01:30:31.756ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5232025-03-25T01:30:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5242025-03-25T01:30:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
5252025-03-25T01:30:31.757ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
5262025-03-25T01:30:31.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
5272025-03-25T01:30:31.760ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-25 01:20:31.740889237 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
5282025-03-25T01:30:31.760ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 19.742552ms iteration = 12 status = {"expiration":"2025-03-25T01:20:31.740889237Z","pruned":[]}
5292025-03-25T01:30:31.760ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1535, tv_nsec: 470947686 } })
5302025-03-25T01:30:31.761ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.932885ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
5312025-03-25T01:30:31.761ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5322025-03-25T01:30:31.761ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
5332025-03-25T01:30:31.761ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 20.441053ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
5342025-03-25T01:30:31.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
5352025-03-25T01:30:31.764ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5362025-03-25T01:30:31.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
5372025-03-25T01:30:31.767ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5382025-03-25T01:30:31.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5392025-03-25T01:30:31.768ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5402025-03-25T01:30:31.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5412025-03-25T01:30:31.772ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5422025-03-25T01:30:31.773ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1535, tv_nsec: 483071780 } })
5432025-03-25T01:30:31.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5442025-03-25T01:30:31.773ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
5452025-03-25T01:30:31.777ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5462025-03-25T01:30:31.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5472025-03-25T01:30:31.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5482025-03-25T01:30:31.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5492025-03-25T01:30:31.780ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5502025-03-25T01:30:31.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5512025-03-25T01:30:31.783ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5522025-03-25T01:30:31.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5532025-03-25T01:30:31.784ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.675234ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5542025-03-25T01:30:31.787ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5552025-03-25T01:30:31.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5562025-03-25T01:30:31.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5572025-03-25T01:30:31.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
5582025-03-25T01:30:31.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5592025-03-25T01:30:31.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5602025-03-25T01:30:31.826ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5612025-03-25T01:30:31.826ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5622025-03-25T01:30:31.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5632025-03-25T01:30:31.827ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
5642025-03-25T01:30:31.827ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
5652025-03-25T01:30:31.827ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 86.300034ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
5662025-03-25T01:30:31.830ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5672025-03-25T01:30:31.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5682025-03-25T01:30:31.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5692025-03-25T01:30:31.840ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5702025-03-25T01:30:31.841ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5712025-03-25T01:30:31.843ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5722025-03-25T01:30:31.843ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5732025-03-25T01:30:31.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5742025-03-25T01:30:31.843ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5752025-03-25T01:30:31.844ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5762025-03-25T01:30:31.844ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
5772025-03-25T01:30:31.844ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5782025-03-25T01:30:31.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5792025-03-25T01:30:31.846ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5802025-03-25T01:30:31.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5812025-03-25T01:30:31.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
5822025-03-25T01:30:31.847ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
5832025-03-25T01:30:31.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
5842025-03-25T01:30:31.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5852025-03-25T01:30:31.847ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 106.670866ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
5862025-03-25T01:30:31.848ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 106.988582ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
5872025-03-25T01:30:31.848ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5882025-03-25T01:30:31.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5892025-03-25T01:30:31.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5902025-03-25T01:30:31.850ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
5912025-03-25T01:30:31.850ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
5922025-03-25T01:30:31.850ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
5932025-03-25T01:30:31.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5942025-03-25T01:30:31.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5952025-03-25T01:30:31.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
5962025-03-25T01:30:31.855ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5972025-03-25T01:30:31.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5982025-03-25T01:30:31.856ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5992025-03-25T01:30:31.856ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
6002025-03-25T01:30:31.856ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 115.579413ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
6012025-03-25T01:30:31.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6022025-03-25T01:30:31.861ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6032025-03-25T01:30:31.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6042025-03-25T01:30:31.862ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
6052025-03-25T01:30:31.862ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 121.478518ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
6062025-03-25T01:30:31.862ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6072025-03-25T01:30:31.864ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6082025-03-25T01:30:31.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6092025-03-25T01:30:31.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6102025-03-25T01:30:31.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6112025-03-25T01:30:31.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6122025-03-25T01:30:31.865ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6132025-03-25T01:30:31.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6142025-03-25T01:30:31.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 1768bc4c-c9ec-4fc7-a560-5ef338d49e99, lookup_type: ById(1768bc4c-c9ec-4fc7-a560-5ef338d49e99) } result = Ok(())
6152025-03-25T01:30:31.868ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6162025-03-25T01:30:31.868ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
6172025-03-25T01:30:31.868ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 127.84353ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
6182025-03-25T01:30:31.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6192025-03-25T01:30:31.879ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 138.693875ms iteration = 23 status = 0
6202025-03-25T01:30:31.899ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
6212025-03-25T01:30:31.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
6222025-03-25T01:30:31.904ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 163.091994ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6232025-03-25T01:30:31.913ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
6242025-03-25T01:30:31.913ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 172.214524ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
6252025-03-25T01:30:31.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6262025-03-25T01:30:31.916ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 175.695114ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6272025-03-25T01:30:31.916ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
6282025-03-25T01:30:31.916ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
6292025-03-25T01:30:31.916ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
6302025-03-25T01:30:31.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6312025-03-25T01:30:31.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6322025-03-25T01:30:31.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6332025-03-25T01:30:31.917ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6342025-03-25T01:30:31.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6352025-03-25T01:30:31.918ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6362025-03-25T01:30:31.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6372025-03-25T01:30:31.920ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6382025-03-25T01:30:31.920ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6392025-03-25T01:30:31.920ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6402025-03-25T01:30:31.920ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6412025-03-25T01:30:31.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
6422025-03-25T01:30:31.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6432025-03-25T01:30:31.920ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
6442025-03-25T01:30:31.920ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6452025-03-25T01:30:31.921ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
6462025-03-25T01:30:31.921ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "97a7967b-bf4d-4366-9f61-cab39be474bb", "content-length": "26", "date": "Tue, 25 Mar 2025 01:30:31 GMT"} })
6472025-03-25T01:30:31.921ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6482025-03-25T01:30:31.922ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6492025-03-25T01:30:31.924ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
6502025-03-25T01:30:31.924ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6512025-03-25T01:30:31.924ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6522025-03-25T01:30:31.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6532025-03-25T01:30:31.930ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
6542025-03-25T01:30:31.930ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60210834-8447-4faf-a5df-01a572eb2d5e dns_zone_name = oxide.test found_so_far = 0 version = 2
6552025-03-25T01:30:31.930ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6562025-03-25T01:30:31.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6572025-03-25T01:30:31.935ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6582025-03-25T01:30:31.939ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
6592025-03-25T01:30:31.939ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6602025-03-25T01:30:31.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
6612025-03-25T01:30:31.941ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60210834-8447-4faf-a5df-01a572eb2d5e dns_zone_name = oxide.test found_so_far = 1 version = 2
6622025-03-25T01:30:31.941ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
6632025-03-25T01:30:31.941ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-25 01:20:07.107344 UTC dns_group = external generation = 2
6642025-03-25T01:30:31.941ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 201.017167ms iteration = 12 status = {"generation":2}
6652025-03-25T01:30:31.956ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
6662025-03-25T01:30:31.956ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6672025-03-25T01:30:31.957ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca51df26-34c3-469d-9a79-a516f2f39017", "content-length": "26", "date": "Tue, 25 Mar 2025 01:30:31 GMT"} })
6682025-03-25T01:30:31.957ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6692025-03-25T01:30:31.959ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6702025-03-25T01:30:31.960ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6712025-03-25T01:30:31.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6722025-03-25T01:30:31.962ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6732025-03-25T01:30:31.962ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
6742025-03-25T01:30:31.962ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 221.722613ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
6752025-03-25T01:30:31.963ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6762025-03-25T01:30:31.965ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 224.37244ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6772025-03-25T01:30:31.967ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6782025-03-25T01:30:31.967ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6792025-03-25T01:30:31.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6802025-03-25T01:30:31.969ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
6812025-03-25T01:30:31.969ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e54f7e5-0241-40be-bb43-5fe289855c17 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
6822025-03-25T01:30:31.969ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6832025-03-25T01:30:31.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6842025-03-25T01:30:31.971ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6852025-03-25T01:30:31.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
6862025-03-25T01:30:31.974ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6872025-03-25T01:30:31.975ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-25T01:19:30.034299Z, time_modified: 2025-03-25T01:19:30.034299Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, db_silo: Silo { identity: SiloIdentity { id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-25T01:20:07.062083Z, time_modified: 2025-03-25T01:20:07.062083Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, db_silo: Silo { identity: SiloIdentity { id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-25T01:20:07.062083Z, time_modified: 2025-03-25T01:20:07.062083Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175" }] }) }
6882025-03-25T01:30:31.975ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 234.355394ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7","tls_certs":["e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175"]}},"default_endpoint":{"silo_id":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7","tls_certs":["e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
6892025-03-25T01:30:31.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6902025-03-25T01:30:31.975ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
6912025-03-25T01:30:31.978ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e54f7e5-0241-40be-bb43-5fe289855c17 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
6922025-03-25T01:30:31.978ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
6932025-03-25T01:30:31.978ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-25 01:20:06.503139 UTC dns_group = internal generation = 1
6942025-03-25T01:30:31.978ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 237.364706ms iteration = 12 status = {"generation":1}
6952025-03-25T01:30:32.015ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd zone_address = fd00:1122:3344:101::2
6962025-03-25T01:30:32.015ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6972025-03-25T01:30:32.015ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e71fe404-5ff3-4d22-9558-9dee9ed61571", "content-length": "26", "date": "Tue, 25 Mar 2025 01:30:31 GMT"} })
6982025-03-25T01:30:32.015ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6992025-03-25T01:30:32.017ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
7002025-03-25T01:30:32.061ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
7012025-03-25T01:30:32.061ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 320.239513ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7022025-03-25T01:30:32.074ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 334.083621ms iteration = 26 status = {"physical_disks_added":0}
7032025-03-25T01:30:32.092ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cbb2831a-b164-4760-b10a-02225a7f5e81 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
7042025-03-25T01:30:32.092ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cbb2831a-b164-4760-b10a-02225a7f5e81 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
7052025-03-25T01:30:32.095ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cbb2831a-b164-4760-b10a-02225a7f5e81 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
7062025-03-25T01:30:32.096ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
7072025-03-25T01:30:32.102ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7082025-03-25T01:30:32.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7092025-03-25T01:30:32.104ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7102025-03-25T01:30:32.106ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7112025-03-25T01:30:32.106ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7122025-03-25T01:30:32.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7132025-03-25T01:30:32.107ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7142025-03-25T01:30:32.109ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7152025-03-25T01:30:32.109ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7162025-03-25T01:30:32.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7172025-03-25T01:30:32.112ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
7182025-03-25T01:30:32.112ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "af951c5e-e9e6-4f15-99b1-56965d838806", "content-length": "1", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7192025-03-25T01:30:32.114ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7202025-03-25T01:30:32.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7212025-03-25T01:30:32.170ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
7222025-03-25T01:30:32.170ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f89b0c33-a8cd-4c6b-bce0-d7eb6c01c728", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7232025-03-25T01:30:32.170ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7242025-03-25T01:30:32.172ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "04abb6e1-2327-47f3-a793-a5e6481381a3", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7252025-03-25T01:30:32.172ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7262025-03-25T01:30:32.174ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
7272025-03-25T01:30:32.174ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 433.874609ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
7282025-03-25T01:30:32.221ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 480.235982ms iteration = 23 status = {}
7292025-03-25T01:30:32.227ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
7302025-03-25T01:30:32.228ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "b8075e61-dc55-484d-9eb8-9eaa55fed50f", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7312025-03-25T01:30:32.228ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7322025-03-25T01:30:32.230ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "45f64ab1-dfa6-44da-bdeb-01c34540844c", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7332025-03-25T01:30:32.230ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7342025-03-25T01:30:32.230ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
7352025-03-25T01:30:32.230ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7362025-03-25T01:30:32.231ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e97d21b9-3f0a-40c5-a77e-2de3cfcce46c", "content-length": "372", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7372025-03-25T01:30:32.231ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7382025-03-25T01:30:32.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7392025-03-25T01:30:32.231ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "736303e9-fc3c-4c11-addd-f5456c33acbb", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7402025-03-25T01:30:32.232ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7412025-03-25T01:30:32.233ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 492.925734ms iteration = 23 status = null
7422025-03-25T01:30:32.233ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
7432025-03-25T01:30:32.233ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 493.030325ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
7442025-03-25T01:30:32.308ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7452025-03-25T01:30:32.308ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7462025-03-25T01:30:32.308ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6df74828-ffd5-4a56-ace3-7835a2bc1994", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7472025-03-25T01:30:32.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7482025-03-25T01:30:32.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7492025-03-25T01:30:32.311ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7502025-03-25T01:30:32.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7512025-03-25T01:30:32.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7522025-03-25T01:30:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7532025-03-25T01:30:32.313ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1a5160a0-1d63-486b-827e-4b7de2ad2079, rack_id: 16a011db-af85-49cc-8fa6-b468f46a78bd, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(80c078c2-e0e7-4e11-a9de-db9b8488eb2c) } rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
7542025-03-25T01:30:32.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7552025-03-25T01:30:32.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7562025-03-25T01:30:32.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7572025-03-25T01:30:32.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7582025-03-25T01:30:32.318ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7592025-03-25T01:30:32.318ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
7602025-03-25T01:30:32.318ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b07bcffa-5d94-4a55-a409-966f41c7fdee", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7612025-03-25T01:30:32.321ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 580.123181ms iteration = 23 status = {}
7622025-03-25T01:30:32.323ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
7632025-03-25T01:30:32.324ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7ff48e9-0276-4693-a26b-aa9599a901e7", "content-length": "134", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7642025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
7652025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 80c078c2-e0e7-4e11-a9de-db9b8488eb2c
7662025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
7672025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
7682025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7692025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7702025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7712025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7722025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
7732025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
7742025-03-25T01:30:32.324ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
7752025-03-25T01:30:32.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7762025-03-25T01:30:32.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7772025-03-25T01:30:32.328ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
7782025-03-25T01:30:32.328ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f19aec2-561c-4289-9426-9b91a156d591", "content-length": "2", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7792025-03-25T01:30:32.328ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
7802025-03-25T01:30:32.328ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5b4a601-2fff-4f88-ad99-761319c9125e", "content-length": "51", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7812025-03-25T01:30:32.328ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
7822025-03-25T01:30:32.328ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
7832025-03-25T01:30:32.373ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7842025-03-25T01:30:32.442ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "570961b0-f052-4b6e-9d0b-88ce58a973cf", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7852025-03-25T01:30:32.442ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch = Switch0
7862025-03-25T01:30:32.442ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
7872025-03-25T01:30:32.443ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f57052ff-cb03-484b-85f3-4c4a95aa0b1b", "content-length": "682", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7882025-03-25T01:30:32.443ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7892025-03-25T01:30:32.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7902025-03-25T01:30:32.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7912025-03-25T01:30:32.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7922025-03-25T01:30:32.448ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbe0cf33-3bd4-4379-8ad5-207e21eedbd8", "content-length": "691", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
7932025-03-25T01:30:32.448ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
7942025-03-25T01:30:32.448ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
7952025-03-25T01:30:32.448ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
7962025-03-25T01:30:32.449ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 708.994955ms iteration = 23 status = {}
7972025-03-25T01:30:32.450ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7982025-03-25T01:30:32.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7992025-03-25T01:30:32.453ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8002025-03-25T01:30:32.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8012025-03-25T01:30:32.459ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 719.311512ms iteration = 23 status = {}
8022025-03-25T01:30:32.503ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
8032025-03-25T01:30:32.503ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
8042025-03-25T01:30:32.534ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
8052025-03-25T01:30:32.534ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:36136
8062025-03-25T01:30:32.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:36136 req_id = fb6af6bd-adbc-4547-8abe-bf2815b6023f roles = RoleSet { roles: {} } uri = /metrics/producers
8072025-03-25T01:30:32.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:36136 req_id = fb6af6bd-adbc-4547-8abe-bf2815b6023f resource = Database result = Ok(()) uri = /metrics/producers
8082025-03-25T01:30:32.587ZINFOnexus: assigned collector to new producer collector_id = a9f260fd-b911-4f87-b07b-84934591a3eb file = nexus/src/app/oximeter.rs:76 producer_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
8092025-03-25T01:30:32.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 52699 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:36136 req_id = fb6af6bd-adbc-4547-8abe-bf2815b6023f response_code = 201 uri = /metrics/producers
8102025-03-25T01:30:32.587ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "fb6af6bd-adbc-4547-8abe-bf2815b6023f", "content-length": "41", "date": "Tue, 25 Mar 2025 01:30:32 GMT"} })
8112025-03-25T01:30:32.589ZDEBGnexus (producer-registration-task): registered with nexus successfully
8122025-03-25T01:30:32.589ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
8132025-03-25T01:30:33.118ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7610a81f-399c-46a9-860a-13305d7f9251 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8142025-03-25T01:30:33.118ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7610a81f-399c-46a9-860a-13305d7f9251 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8152025-03-25T01:30:33.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1798 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7610a81f-399c-46a9-860a-13305d7f9251 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8162025-03-25T01:30:34.139ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ffa57064-8d30-4075-ad21-9dde6cda6f9f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8172025-03-25T01:30:34.139ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ffa57064-8d30-4075-ad21-9dde6cda6f9f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8182025-03-25T01:30:34.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2594 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ffa57064-8d30-4075-ad21-9dde6cda6f9f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8192025-03-25T01:30:35.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0f85df79-b451-46bd-a446-fe8b036c3226 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8202025-03-25T01:30:35.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0f85df79-b451-46bd-a446-fe8b036c3226 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8212025-03-25T01:30:35.168ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4944 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0f85df79-b451-46bd-a446-fe8b036c3226 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8222025-03-25T01:30:35.767ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 505 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = aa8481b2-a388-4b6b-91f9-5bce324cbf7d response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
8232025-03-25T01:30:36.190ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5099ba96-0d2e-4cec-86a0-e77d7d37f365 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8242025-03-25T01:30:36.190ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5099ba96-0d2e-4cec-86a0-e77d7d37f365 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8252025-03-25T01:30:36.192ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2328 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5099ba96-0d2e-4cec-86a0-e77d7d37f365 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8262025-03-25T01:30:37.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6153cece-d1d6-4fd0-8d9e-c881fb620a6f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8272025-03-25T01:30:37.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6153cece-d1d6-4fd0-8d9e-c881fb620a6f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8282025-03-25T01:30:37.214ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1927 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6153cece-d1d6-4fd0-8d9e-c881fb620a6f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8292025-03-25T01:30:38.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b70ce1d4-c386-4bc6-9ae2-849ea03a5726 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8302025-03-25T01:30:38.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b70ce1d4-c386-4bc6-9ae2-849ea03a5726 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8312025-03-25T01:30:38.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2224 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b70ce1d4-c386-4bc6-9ae2-849ea03a5726 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8322025-03-25T01:30:39.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 26860330-f2cf-4b4a-be33-f5abaf4fd2b7 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8332025-03-25T01:30:39.257ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 26860330-f2cf-4b4a-be33-f5abaf4fd2b7 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8342025-03-25T01:30:39.259ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 26860330-f2cf-4b4a-be33-f5abaf4fd2b7 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8352025-03-25T01:30:40.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5f0c7ba2-c0d0-4269-8199-3333aa09da88 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8362025-03-25T01:30:40.279ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5f0c7ba2-c0d0-4269-8199-3333aa09da88 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8372025-03-25T01:30:40.280ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2391 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 5f0c7ba2-c0d0-4269-8199-3333aa09da88 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8382025-03-25T01:30:41.303ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0f4d9789-0ee0-4b7b-9c91-01a57d3eb1be roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8392025-03-25T01:30:41.303ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0f4d9789-0ee0-4b7b-9c91-01a57d3eb1be resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8402025-03-25T01:30:41.305ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2436 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0f4d9789-0ee0-4b7b-9c91-01a57d3eb1be response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8412025-03-25T01:30:41.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
8422025-03-25T01:30:41.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8432025-03-25T01:30:41.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8442025-03-25T01:30:41.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
8452025-03-25T01:30:41.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8462025-03-25T01:30:41.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8472025-03-25T01:30:41.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8482025-03-25T01:30:41.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8492025-03-25T01:30:41.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8502025-03-25T01:30:41.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8512025-03-25T01:30:41.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8522025-03-25T01:30:41.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8532025-03-25T01:30:41.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8542025-03-25T01:30:41.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
8552025-03-25T01:30:41.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8562025-03-25T01:30:41.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8572025-03-25T01:30:41.761ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
8582025-03-25T01:30:41.761ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.91841ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
8592025-03-25T01:30:41.772ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8602025-03-25T01:30:41.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8612025-03-25T01:30:41.801ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 60.523626ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8622025-03-25T01:30:42.325ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b8781ff6-1a1c-4ee7-b0b8-395f4b54d4ca roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8632025-03-25T01:30:42.326ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b8781ff6-1a1c-4ee7-b0b8-395f4b54d4ca resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8642025-03-25T01:30:42.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2120 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b8781ff6-1a1c-4ee7-b0b8-395f4b54d4ca response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8652025-03-25T01:30:43.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 675df8f0-3362-4c28-bafa-771158bb49df roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8662025-03-25T01:30:43.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 675df8f0-3362-4c28-bafa-771158bb49df resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8672025-03-25T01:30:43.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2400 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 675df8f0-3362-4c28-bafa-771158bb49df response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8682025-03-25T01:30:44.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 194566c2-1bfd-46f9-badd-e56e914dfc58 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8692025-03-25T01:30:44.372ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 194566c2-1bfd-46f9-badd-e56e914dfc58 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8702025-03-25T01:30:44.374ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2188 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 194566c2-1bfd-46f9-badd-e56e914dfc58 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8712025-03-25T01:30:45.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 94dc3c58-2a54-4d4d-8573-ebf91bf78fc9 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8722025-03-25T01:30:45.394ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 94dc3c58-2a54-4d4d-8573-ebf91bf78fc9 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8732025-03-25T01:30:45.396ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2014 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 94dc3c58-2a54-4d4d-8573-ebf91bf78fc9 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8742025-03-25T01:30:45.767ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 575 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 46ed1e05-536a-4104-9629-c422f14306e9 response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
8752025-03-25T01:30:46.418ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 075cf472-af63-4ca2-870c-6382a498cdb8 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8762025-03-25T01:30:46.418ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 075cf472-af63-4ca2-870c-6382a498cdb8 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8772025-03-25T01:30:46.420ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2189 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 075cf472-af63-4ca2-870c-6382a498cdb8 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8782025-03-25T01:30:47.442ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 60aa35a8-cd17-4597-af2c-aafc4e84049e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8792025-03-25T01:30:47.442ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 60aa35a8-cd17-4597-af2c-aafc4e84049e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8802025-03-25T01:30:47.444ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2136 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 60aa35a8-cd17-4597-af2c-aafc4e84049e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8812025-03-25T01:30:48.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 31dc1337-ed38-4b25-b5a3-7b87b1eb4898 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8822025-03-25T01:30:48.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 31dc1337-ed38-4b25-b5a3-7b87b1eb4898 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8832025-03-25T01:30:48.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2015 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 31dc1337-ed38-4b25-b5a3-7b87b1eb4898 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8842025-03-25T01:30:49.487ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 24fc4bfd-40f2-498f-b5df-a49183d7a05b roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8852025-03-25T01:30:49.487ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 24fc4bfd-40f2-498f-b5df-a49183d7a05b resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8862025-03-25T01:30:49.489ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1918 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 24fc4bfd-40f2-498f-b5df-a49183d7a05b response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8872025-03-25T01:30:50.511ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c480b472-913a-4f05-ba2f-335a550e41be roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8882025-03-25T01:30:50.511ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c480b472-913a-4f05-ba2f-335a550e41be resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8892025-03-25T01:30:50.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2163 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c480b472-913a-4f05-ba2f-335a550e41be response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8902025-03-25T01:30:51.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f925b9c8-bfa5-419f-b7de-2b9bee60aaee roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8912025-03-25T01:30:51.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f925b9c8-bfa5-419f-b7de-2b9bee60aaee resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8922025-03-25T01:30:51.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2335 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f925b9c8-bfa5-419f-b7de-2b9bee60aaee response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
8932025-03-25T01:30:51.741ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
8942025-03-25T01:30:51.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8952025-03-25T01:30:51.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8962025-03-25T01:30:51.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
8972025-03-25T01:30:51.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8982025-03-25T01:30:51.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8992025-03-25T01:30:51.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9002025-03-25T01:30:51.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9012025-03-25T01:30:51.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9022025-03-25T01:30:51.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9032025-03-25T01:30:51.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9042025-03-25T01:30:51.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9052025-03-25T01:30:51.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9062025-03-25T01:30:51.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
9072025-03-25T01:30:51.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9082025-03-25T01:30:51.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9092025-03-25T01:30:51.774ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
9102025-03-25T01:30:51.774ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.159234ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
9112025-03-25T01:30:51.790ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9122025-03-25T01:30:51.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9132025-03-25T01:30:51.792ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 50.572403ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9142025-03-25T01:30:52.561ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 68315561-d549-4f92-9eff-242600324f2e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9152025-03-25T01:30:52.561ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 68315561-d549-4f92-9eff-242600324f2e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9162025-03-25T01:30:52.563ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2190 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 68315561-d549-4f92-9eff-242600324f2e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9172025-03-25T01:30:53.585ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7c75e873-f132-435f-b396-d580f48ca82a roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9182025-03-25T01:30:53.585ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7c75e873-f132-435f-b396-d580f48ca82a resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9192025-03-25T01:30:53.587ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2376 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7c75e873-f132-435f-b396-d580f48ca82a response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9202025-03-25T01:30:54.610ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 748c3634-dbc3-4105-b5eb-8ac5960dfb2d roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9212025-03-25T01:30:54.610ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 748c3634-dbc3-4105-b5eb-8ac5960dfb2d resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9222025-03-25T01:30:54.612ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2085 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 748c3634-dbc3-4105-b5eb-8ac5960dfb2d response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9232025-03-25T01:30:55.633ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = aecff442-ae41-4206-a71d-26c8733cb40f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9242025-03-25T01:30:55.634ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = aecff442-ae41-4206-a71d-26c8733cb40f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9252025-03-25T01:30:55.635ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1641 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = aecff442-ae41-4206-a71d-26c8733cb40f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9262025-03-25T01:30:55.767ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 505 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = bb830327-d50d-41bf-a07a-b312e17853bb response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
9272025-03-25T01:30:56.655ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dac24fa8-e7db-4e9d-a1ec-9d818d5fc70f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9282025-03-25T01:30:56.655ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dac24fa8-e7db-4e9d-a1ec-9d818d5fc70f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9292025-03-25T01:30:56.657ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2412 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dac24fa8-e7db-4e9d-a1ec-9d818d5fc70f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9302025-03-25T01:30:57.675ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4ab2be33-299b-4298-8d60-cb5ff0bad154 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9312025-03-25T01:30:57.676ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4ab2be33-299b-4298-8d60-cb5ff0bad154 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9322025-03-25T01:30:57.678ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2728 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4ab2be33-299b-4298-8d60-cb5ff0bad154 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9332025-03-25T01:30:58.464ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 845e2001-8c8b-4617-adcc-87e11954ff6b uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-03-25T01:30:58.442353793Z }
9342025-03-25T01:30:58.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 845e2001-8c8b-4617-adcc-87e11954ff6b roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
9352025-03-25T01:30:58.465ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 845e2001-8c8b-4617-adcc-87e11954ff6b resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
9362025-03-25T01:30:58.513ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 70789 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 845e2001-8c8b-4617-adcc-87e11954ff6b response_code = 204 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
9372025-03-25T01:30:58.526ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ff1426bf-4fa8-4ae3-8111-e82a96d5b3d6 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-25T01:30:58.525821557Z }
9382025-03-25T01:30:58.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ff1426bf-4fa8-4ae3-8111-e82a96d5b3d6 roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
9392025-03-25T01:30:58.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ff1426bf-4fa8-4ae3-8111-e82a96d5b3d6 resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
9402025-03-25T01:30:58.533ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 7667 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ff1426bf-4fa8-4ae3-8111-e82a96d5b3d6 response_code = 204 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
9412025-03-25T01:30:58.700ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 263d17ba-ee39-4ebb-a382-88a7a67b5b2b roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9422025-03-25T01:30:58.700ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 263d17ba-ee39-4ebb-a382-88a7a67b5b2b resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9432025-03-25T01:30:58.701ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1619 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 263d17ba-ee39-4ebb-a382-88a7a67b5b2b response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9442025-03-25T01:30:59.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9b86daec-af8a-471e-8a80-ff591448adba roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9452025-03-25T01:30:59.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9b86daec-af8a-471e-8a80-ff591448adba resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9462025-03-25T01:30:59.725ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2334 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9b86daec-af8a-471e-8a80-ff591448adba response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9472025-03-25T01:31:00.744ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2b6d1c7a-01c4-47cf-954a-a898420759ad roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9482025-03-25T01:31:00.744ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2b6d1c7a-01c4-47cf-954a-a898420759ad resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9492025-03-25T01:31:00.746ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1955 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2b6d1c7a-01c4-47cf-954a-a898420759ad response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
9502025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
9512025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
9522025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9532025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
9542025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
9552025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
9562025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
9572025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
9582025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9592025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
9602025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9612025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
9622025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
9632025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
9642025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9652025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 24 reason = Timeout
9662025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
9672025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9682025-03-25T01:31:01.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9692025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9702025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9712025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
9722025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9732025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9742025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9752025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
9762025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
9772025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
9782025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
9792025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
9802025-03-25T01:31:01.741ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
9812025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9822025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
9832025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9842025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
9852025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
9862025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9872025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
9882025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9892025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
9902025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
9912025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
9922025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
9932025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
9942025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
9952025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
9962025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
9972025-03-25T01:31:01.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9982025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9992025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10002025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
10012025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
10022025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10032025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10042025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
10052025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10062025-03-25T01:31:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10072025-03-25T01:31:01.743ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10082025-03-25T01:31:01.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10092025-03-25T01:31:01.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10102025-03-25T01:31:01.745ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.043663ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10112025-03-25T01:31:01.746ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1565, tv_nsec: 456523910 } })
10122025-03-25T01:31:01.747ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
10132025-03-25T01:31:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
10142025-03-25T01:31:01.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10152025-03-25T01:31:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10162025-03-25T01:31:01.748ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
10172025-03-25T01:31:01.748ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10182025-03-25T01:31:01.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10192025-03-25T01:31:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10202025-03-25T01:31:01.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10212025-03-25T01:31:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10222025-03-25T01:31:01.751ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10232025-03-25T01:31:01.751ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1565, tv_nsec: 461167526 } })
10242025-03-25T01:31:01.751ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
10252025-03-25T01:31:01.752ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10262025-03-25T01:31:01.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10272025-03-25T01:31:01.753ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10282025-03-25T01:31:01.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10292025-03-25T01:31:01.754ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10302025-03-25T01:31:01.755ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10312025-03-25T01:31:01.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10322025-03-25T01:31:01.756ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10332025-03-25T01:31:01.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
10342025-03-25T01:31:01.759ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10352025-03-25T01:31:01.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
10362025-03-25T01:31:01.762ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10372025-03-25T01:31:01.762ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
10382025-03-25T01:31:01.762ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10392025-03-25T01:31:01.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10402025-03-25T01:31:01.764ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10412025-03-25T01:31:01.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10422025-03-25T01:31:01.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 3a52af07-1916-4180-8201-23f56dc85536 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
10432025-03-25T01:31:01.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10442025-03-25T01:31:01.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10452025-03-25T01:31:01.768ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1565, tv_nsec: 477984307 } })
10462025-03-25T01:31:01.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 3a52af07-1916-4180-8201-23f56dc85536 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
10472025-03-25T01:31:01.768ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
10482025-03-25T01:31:01.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10492025-03-25T01:31:01.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10502025-03-25T01:31:01.769ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10512025-03-25T01:31:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10522025-03-25T01:31:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10532025-03-25T01:31:01.772ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10542025-03-25T01:31:01.772ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10552025-03-25T01:31:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10562025-03-25T01:31:01.778ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 10913 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 3a52af07-1916-4180-8201-23f56dc85536 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
10572025-03-25T01:31:01.782ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10582025-03-25T01:31:01.784ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
10592025-03-25T01:31:01.784ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10602025-03-25T01:31:01.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10612025-03-25T01:31:01.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10622025-03-25T01:31:01.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
10632025-03-25T01:31:01.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10642025-03-25T01:31:01.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10652025-03-25T01:31:01.808ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
10662025-03-25T01:31:01.809ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 68.404028ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
10672025-03-25T01:31:01.812ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10682025-03-25T01:31:01.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
10692025-03-25T01:31:01.815ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10702025-03-25T01:31:01.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10712025-03-25T01:31:01.829ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 87.767086ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
10722025-03-25T01:31:01.832ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 90.424954ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
10732025-03-25T01:31:01.834ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 92.433333ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
10742025-03-25T01:31:01.834ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10752025-03-25T01:31:01.834ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10762025-03-25T01:31:01.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
10772025-03-25T01:31:01.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10782025-03-25T01:31:01.836ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 96.037175ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10792025-03-25T01:31:01.837ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10802025-03-25T01:31:01.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10812025-03-25T01:31:01.838ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10822025-03-25T01:31:01.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10832025-03-25T01:31:01.839ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
10842025-03-25T01:31:01.839ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
10852025-03-25T01:31:01.839ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10862025-03-25T01:31:01.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10872025-03-25T01:31:01.840ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10882025-03-25T01:31:01.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10892025-03-25T01:31:01.841ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10902025-03-25T01:31:01.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
10912025-03-25T01:31:01.842ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 100.891304ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10922025-03-25T01:31:01.843ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10932025-03-25T01:31:01.845ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
10942025-03-25T01:31:01.845ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10952025-03-25T01:31:01.845ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
10962025-03-25T01:31:01.845ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
10972025-03-25T01:31:01.845ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6b14cb2-9d8a-419d-9e78-4c8b70094677", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
10982025-03-25T01:31:01.845ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
10992025-03-25T01:31:01.846ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6157c2fd-726f-4394-8d93-21fe44f62813", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11002025-03-25T01:31:01.846ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11012025-03-25T01:31:01.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 1768bc4c-c9ec-4fc7-a560-5ef338d49e99, lookup_type: ById(1768bc4c-c9ec-4fc7-a560-5ef338d49e99) } result = Ok(())
11022025-03-25T01:31:01.847ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11032025-03-25T01:31:01.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11042025-03-25T01:31:01.848ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11052025-03-25T01:31:01.848ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11062025-03-25T01:31:01.849ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd zone_address = fd00:1122:3344:101::2
11072025-03-25T01:31:01.849ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11082025-03-25T01:31:01.849ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e537d91-3b90-45e2-943f-af578650ba54", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11092025-03-25T01:31:01.849ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11102025-03-25T01:31:01.849ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 108.034456ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11112025-03-25T01:31:01.849ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 109.275143ms iteration = 24 status = 0
11122025-03-25T01:31:01.851ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
11132025-03-25T01:31:01.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 145.807346ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11142025-03-25T01:31:01.895ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11152025-03-25T01:31:01.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11162025-03-25T01:31:01.898ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
11172025-03-25T01:31:01.898ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
11182025-03-25T01:31:01.898ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 156.730313ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
11192025-03-25T01:31:01.900ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11202025-03-25T01:31:01.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11212025-03-25T01:31:01.938ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 197.993433ms iteration = 27 status = {"physical_disks_added":0}
11222025-03-25T01:31:01.969ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
11232025-03-25T01:31:01.969ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "b24313d6-1f31-467a-89b5-d89d81f7fee9", "content-length": "1", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11242025-03-25T01:31:01.970ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
11252025-03-25T01:31:01.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
11262025-03-25T01:31:01.983ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70/state
11272025-03-25T01:31:01.984ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "baabb18e-c9ef-4fc2-b291-7881f23ad68a", "content-length": "130", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11282025-03-25T01:31:01.985ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11292025-03-25T01:31:01.986ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 sled_id = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-25T01:30:58.525821557Z }, migration_in: None, migration_out: None } vmm_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70
11302025-03-25T01:31:01.986ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2088 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 sled_id = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 vmm_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-03-25T01:30:58.525821557Z }
11312025-03-25T01:31:01.986ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 roles = RoleSet { roles: {} } sled_id = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 vmm_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70
11322025-03-25T01:31:01.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 resource = Database result = Ok(()) sled_id = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 vmm_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70
11332025-03-25T01:31:01.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11342025-03-25T01:31:01.987ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11352025-03-25T01:31:01.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11362025-03-25T01:31:01.988ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 247.751735ms iteration = 24 status = {}
11372025-03-25T01:31:02.012ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
11382025-03-25T01:31:02.012ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11392025-03-25T01:31:02.012ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "df923eba-41a8-4656-9a85-453644ec6473", "content-length": "585", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11402025-03-25T01:31:02.012ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11412025-03-25T01:31:02.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11422025-03-25T01:31:02.015ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 274.582599ms iteration = 24 status = null
11432025-03-25T01:31:02.068ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
11442025-03-25T01:31:02.068ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
11452025-03-25T01:31:02.068ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 328.084305ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11462025-03-25T01:31:02.070ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 328.691793ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
11472025-03-25T01:31:02.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11482025-03-25T01:31:02.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11492025-03-25T01:31:02.103ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1a5160a0-1d63-486b-827e-4b7de2ad2079, rack_id: 16a011db-af85-49cc-8fa6-b468f46a78bd, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(80c078c2-e0e7-4e11-a9de-db9b8488eb2c) } rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
11502025-03-25T01:31:02.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11512025-03-25T01:31:02.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11522025-03-25T01:31:02.103ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5ff385df-8a4d-4b05-95b1-36d92835d6d3: {10.151.1.176: {3a1bce35-9900-47e8-b587-abca91e7ba03}, 10.151.1.175: {3a1bce35-9900-47e8-b587-abca91e7ba03}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
11532025-03-25T01:31:02.103ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11542025-03-25T01:31:02.104ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6d4a1ca3-5cad-4bc1-9f99-9de3c4bb55e8", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11552025-03-25T01:31:02.104ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11562025-03-25T01:31:02.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11572025-03-25T01:31:02.110ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11582025-03-25T01:31:02.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11592025-03-25T01:31:02.115ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11602025-03-25T01:31:02.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11612025-03-25T01:31:02.128ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
11622025-03-25T01:31:02.129ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "a83526cf-2cc7-44e4-9cc6-e01ccf5d4af2", "content-length": "134", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11632025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
11642025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 80c078c2-e0e7-4e11-a9de-db9b8488eb2c
11652025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
11662025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
11672025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11682025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11692025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11702025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
11712025-03-25T01:31:02.129ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
11722025-03-25T01:31:02.130ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
11732025-03-25T01:31:02.130ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
11742025-03-25T01:31:02.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11752025-03-25T01:31:02.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11762025-03-25T01:31:02.131ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11772025-03-25T01:31:02.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11782025-03-25T01:31:02.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
11792025-03-25T01:31:02.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ffffca-4eab-484b-a1e8-1f57197834cb", "content-length": "2", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11802025-03-25T01:31:02.133ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
11812025-03-25T01:31:02.133ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "f684368e-fe01-4ff8-bec2-a28a079a00a5", "content-length": "51", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
11822025-03-25T01:31:02.133ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
11832025-03-25T01:31:02.133ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
11842025-03-25T01:31:02.134ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11852025-03-25T01:31:02.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11862025-03-25T01:31:02.139ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11872025-03-25T01:31:02.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11882025-03-25T01:31:02.148ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
11892025-03-25T01:31:02.148ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
11902025-03-25T01:31:02.148ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11912025-03-25T01:31:02.181ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11922025-03-25T01:31:02.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11932025-03-25T01:31:02.184ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11942025-03-25T01:31:02.186ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11952025-03-25T01:31:02.186ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11962025-03-25T01:31:02.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11972025-03-25T01:31:02.188ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11982025-03-25T01:31:02.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
11992025-03-25T01:31:02.190ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12002025-03-25T01:31:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12012025-03-25T01:31:02.226ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12022025-03-25T01:31:02.251ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9e2f04ca-0562-4d85-9ba7-026485d5f33e", "date": "Tue, 25 Mar 2025 01:31:01 GMT"} })
12032025-03-25T01:31:02.251ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch = Switch0
12042025-03-25T01:31:02.251ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
12052025-03-25T01:31:02.252ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddbc05ed-abba-4f58-9b74-8ce439b548a0", "content-length": "682", "date": "Tue, 25 Mar 2025 01:31:02 GMT"} })
12062025-03-25T01:31:02.252ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12072025-03-25T01:31:02.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12082025-03-25T01:31:02.256ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12092025-03-25T01:31:02.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12102025-03-25T01:31:02.258ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc435790-2724-478a-b13e-fa1ef5714d30", "content-length": "811", "date": "Tue, 25 Mar 2025 01:31:02 GMT"} })
12112025-03-25T01:31:02.258ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(11810973) }} sled = buskin
12122025-03-25T01:31:02.258ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
12132025-03-25T01:31:02.258ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
12142025-03-25T01:31:02.259ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e21e58f8-1717-43df-8156-716b90bf9388", "date": "Tue, 25 Mar 2025 01:31:02 GMT"} })
12152025-03-25T01:31:02.260ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 518.860384ms iteration = 24 status = {}
12162025-03-25T01:31:02.261ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 520.665761ms iteration = 24 status = {}
12172025-03-25T01:31:02.261ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12182025-03-25T01:31:02.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12192025-03-25T01:31:02.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12202025-03-25T01:31:02.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12212025-03-25T01:31:02.271ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 531.220191ms iteration = 24 status = {}
12222025-03-25T01:31:02.820ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4fc97e9c-4fcb-427b-bb22-f55349398143 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12232025-03-25T01:31:02.820ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4fc97e9c-4fcb-427b-bb22-f55349398143 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12242025-03-25T01:31:02.822ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2412 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4fc97e9c-4fcb-427b-bb22-f55349398143 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12252025-03-25T01:31:03.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 52ea93b7-8e4f-4d86-91e3-a8b80e5b9785 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12262025-03-25T01:31:03.845ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 52ea93b7-8e4f-4d86-91e3-a8b80e5b9785 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12272025-03-25T01:31:03.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2091 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 52ea93b7-8e4f-4d86-91e3-a8b80e5b9785 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12282025-03-25T01:31:04.868ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7a0ef56d-b25e-489b-b4e6-b22d38bf66d3 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12292025-03-25T01:31:04.868ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7a0ef56d-b25e-489b-b4e6-b22d38bf66d3 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12302025-03-25T01:31:04.870ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2671 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7a0ef56d-b25e-489b-b4e6-b22d38bf66d3 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12312025-03-25T01:31:05.771ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3524 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = d1769269-5022-4c10-ad27-f204bf7bbf0b response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
12322025-03-25T01:31:05.892ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 778c63e3-f00e-4331-be3a-fd23dcc53c64 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12332025-03-25T01:31:05.892ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 778c63e3-f00e-4331-be3a-fd23dcc53c64 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12342025-03-25T01:31:05.894ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 778c63e3-f00e-4331-be3a-fd23dcc53c64 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12352025-03-25T01:31:06.915ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = d261187a-a2b0-422b-bddf-19423a6cb97d roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12362025-03-25T01:31:06.915ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = d261187a-a2b0-422b-bddf-19423a6cb97d resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12372025-03-25T01:31:06.917ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2534 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = d261187a-a2b0-422b-bddf-19423a6cb97d response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12382025-03-25T01:31:07.827ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dc38b20e-dc23-4b7c-b0f4-7cdfc4dd6a08 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-03-25T01:31:07.826985506Z }
12392025-03-25T01:31:07.827ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dc38b20e-dc23-4b7c-b0f4-7cdfc4dd6a08 roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12402025-03-25T01:31:07.827ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dc38b20e-dc23-4b7c-b0f4-7cdfc4dd6a08 resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12412025-03-25T01:31:07.839ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 12667 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dc38b20e-dc23-4b7c-b0f4-7cdfc4dd6a08 response_code = 204 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12422025-03-25T01:31:07.938ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4dafaf95-2f66-48d5-a851-8e9555b29a32 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12432025-03-25T01:31:07.939ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4dafaf95-2f66-48d5-a851-8e9555b29a32 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12442025-03-25T01:31:07.940ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1656 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4dafaf95-2f66-48d5-a851-8e9555b29a32 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12452025-03-25T01:31:08.965ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 13f383c2-f5be-4e13-8334-93a87fa89eef roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12462025-03-25T01:31:08.965ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 13f383c2-f5be-4e13-8334-93a87fa89eef resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12472025-03-25T01:31:08.967ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2289 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 13f383c2-f5be-4e13-8334-93a87fa89eef response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12482025-03-25T01:31:09.986ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 07f3937a-8fa9-4b61-9f38-cb3fb2c844e2 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12492025-03-25T01:31:09.986ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 07f3937a-8fa9-4b61-9f38-cb3fb2c844e2 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12502025-03-25T01:31:09.989ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2454 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 07f3937a-8fa9-4b61-9f38-cb3fb2c844e2 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
12512025-03-25T01:31:10.263ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-25T01:31:10.263196485Z }
12522025-03-25T01:31:10.263ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12532025-03-25T01:31:10.263ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12542025-03-25T01:31:10.273ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_needs_update = true
12552025-03-25T01:31:10.273ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12562025-03-25T01:31:10.273ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12572025-03-25T01:31:10.275ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12582025-03-25T01:31:10.276ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12592025-03-25T01:31:10.278ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12602025-03-25T01:31:10.278ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12612025-03-25T01:31:10.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12622025-03-25T01:31:10.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12632025-03-25T01:31:10.282ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12642025-03-25T01:31:10.282ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12652025-03-25T01:31:10.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12662025-03-25T01:31:10.284ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12672025-03-25T01:31:10.285ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12682025-03-25T01:31:10.288ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, lookup_type: ById(9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7) }, key: edb4be7f-ad8b-42c9-b534-94f7c87d25d8, lookup_type: ById(edb4be7f-ad8b-42c9-b534-94f7c87d25d8) }, key: 7970c9b8-9b62-47f6-bc44-40934d16c549, lookup_type: ById(7970c9b8-9b62-47f6-bc44-40934d16c549) } result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12692025-03-25T01:31:10.297ZINFOnexus (dropshot_internal): starting update saga for 7970c9b8-9b62-47f6-bc44-40934d16c549 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1551 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-25T01:31:10.263196485Z }
12702025-03-25T01:31:10.297ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 34028 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb response_code = 204 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12712025-03-25T01:31:10.297ZDEBGnexus (dropshot_internal): preparing instance-update saga for 7970c9b8-9b62-47f6-bc44-40934d16c549... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 09417574-73af-4f1f-bf1e-10d7a43566fb uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12722025-03-25T01:31:10.297ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update
12732025-03-25T01:31:10.298ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2088 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcc233b7-2c1f-4fb7-ade5-c8c581cbd16f uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-03-25T01:31:10.263196485Z }
12742025-03-25T01:31:10.298ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcc233b7-2c1f-4fb7-ade5-c8c581cbd16f roles = RoleSet { roles: {} } uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12752025-03-25T01:31:10.298ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcc233b7-2c1f-4fb7-ade5-c8c581cbd16f resource = Database result = Ok(()) uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12762025-03-25T01:31:10.302ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4217 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcc233b7-2c1f-4fb7-ade5-c8c581cbd16f response_code = 204 uri = /vmms/1a94d6a0-c931-4ad1-aefb-f87808324f70
12772025-03-25T01:31:10.304ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"7970c9b8-9b62-47f6-bc44-40934d16c549","lookup_type":{"ById":"7970c9b8-9b62-47f6-bc44-40934d16c549"},"parent":{"key":"edb4be7f-ad8b-42c9-b534-94f7c87d25d8","lookup_type":{"ById":"edb4be7f-ad8b-42c9-b534-94f7c87d25d8"},"parent":{"key":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7","lookup_type":{"ById":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12782025-03-25T01:31:10.304ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update
12792025-03-25T01:31:10.345ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update
12802025-03-25T01:31:10.345ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12812025-03-25T01:31:10.347ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12822025-03-25T01:31:10.347ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
12832025-03-25T01:31:10.352ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12842025-03-25T01:31:10.352ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
12852025-03-25T01:31:10.356ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12862025-03-25T01:31:10.356ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
12872025-03-25T01:31:10.365ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12882025-03-25T01:31:10.365ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("5e697aaf-7503-48a7-b90c-ddad0ba0273e")) node_id = 0 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
12892025-03-25T01:31:10.369ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
12902025-03-25T01:31:10.369ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
12912025-03-25T01:31:10.385ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update
12922025-03-25T01:31:10.390ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12932025-03-25T01:31:10.390ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12942025-03-25T01:31:10.392ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12952025-03-25T01:31:10.392ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12962025-03-25T01:31:10.394ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12972025-03-25T01:31:10.395ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12982025-03-25T01:31:10.399ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
12992025-03-25T01:31:10.399ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13002025-03-25T01:31:10.401ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13012025-03-25T01:31:10.401ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13022025-03-25T01:31:10.402ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13032025-03-25T01:31:10.402ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13042025-03-25T01:31:10.403ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13052025-03-25T01:31:10.409ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, lookup_type: ById(9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7) }, key: edb4be7f-ad8b-42c9-b534-94f7c87d25d8, lookup_type: ById(edb4be7f-ad8b-42c9-b534-94f7c87d25d8) }, key: 7970c9b8-9b62-47f6-bc44-40934d16c549, lookup_type: ById(7970c9b8-9b62-47f6-bc44-40934d16c549) } result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13062025-03-25T01:31:10.411ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance updater_id = 5e697aaf-7503-48a7-b90c-ddad0ba0273e
13072025-03-25T01:31:10.411ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13082025-03-25T01:31:10.411ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13092025-03-25T01:31:10.426ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance updater_id = 5e697aaf-7503-48a7-b90c-ddad0ba0273e
13102025-03-25T01:31:10.426ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13112025-03-25T01:31:10.427ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance
13122025-03-25T01:31:10.431ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 locked_gen = Generation(Generation(2)) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = LockInstance updater_id = 5e697aaf-7503-48a7-b90c-ddad0ba0273e
13132025-03-25T01:31:10.431ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13142025-03-25T01:31:10.431ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("5e697aaf-7503-48a7-b90c-ddad0ba0273e")}) node_id = 1 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
13152025-03-25T01:31:10.435ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13162025-03-25T01:31:10.435ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
13172025-03-25T01:31:10.443ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13182025-03-25T01:31:10.443ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13192025-03-25T01:31:10.445ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13202025-03-25T01:31:10.445ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13212025-03-25T01:31:10.446ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13222025-03-25T01:31:10.446ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13232025-03-25T01:31:10.447ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13242025-03-25T01:31:10.450ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, lookup_type: ById(9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7) }, key: edb4be7f-ad8b-42c9-b534-94f7c87d25d8, lookup_type: ById(edb4be7f-ad8b-42c9-b534-94f7c87d25d8) }, key: 7970c9b8-9b62-47f6-bc44-40934d16c549, lookup_type: ById(7970c9b8-9b62-47f6-bc44-40934d16c549) } result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13252025-03-25T01:31:10.450ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13262025-03-25T01:31:10.450ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
13272025-03-25T01:31:10.456ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: 1a94d6a0-c931-4ad1-aefb-f87808324f70, time_created: 2025-03-25T01:30:49.594155Z, time_deleted: None, instance_id: 7970c9b8-9b62-47f6-bc44-40934d16c549, sled_id: 1768bc4c-c9ec-4fc7-a560-5ef338d49e99, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2025-03-25T01:31:10.263196Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2025-03-25T01:30:46.773560Z, gen: Generation(Generation(3)), propolis_id: Some(1a94d6a0-c931-4ad1-aefb-f87808324f70), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(1a94d6a0-c931-4ad1-aefb-f87808324f70 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2025-03-25T01:31:10.456853856Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
13282025-03-25T01:31:10.459ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13292025-03-25T01:31:10.459ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"7970c9b8-9b62-47f6-bc44-40934d16c549","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"1a94d6a0-c931-4ad1-aefb-f87808324f70"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"7970c9b8-9b62-47f6-bc44-40934d16c549","lookup_type":{"ById":"7970c9b8-9b62-47f6-bc44-40934d16c549"},"parent":{"key":"edb4be7f-ad8b-42c9-b534-94f7c87d25d8","lookup_type":{"ById":"edb4be7f-ad8b-42c9-b534-94f7c87d25d8"},"parent":{"key":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7","lookup_type":{"ById":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"5e697aaf-7503-48a7-b90c-ddad0ba0273e"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"edb4be7f-ad8b-42c9-b534-94f7c87d25d8","ram_diff":1073741824},"destroy_active_vmm":"1a94d6a0-c931-4ad1-aefb-f87808324f70","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2025-03-25T01:31:10.456853856Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1146 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13302025-03-25T01:31:10.459ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13312025-03-25T01:31:10.464ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13322025-03-25T01:31:10.464ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:967 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13332025-03-25T01:31:10.464ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13342025-03-25T01:31:10.464ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13352025-03-25T01:31:10.467ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13362025-03-25T01:31:10.467ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13372025-03-25T01:31:10.471ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13382025-03-25T01:31:10.471ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13392025-03-25T01:31:10.474ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13402025-03-25T01:31:10.474ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("cac15bc4-0f6d-49fe-bc2e-20866e2ad94c")) node_id = 0 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13412025-03-25T01:31:10.478ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13422025-03-25T01:31:10.478ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13432025-03-25T01:31:10.484ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 parent_lock = UpdaterLock { updater_id: 5e697aaf-7503-48a7-b90c-ddad0ba0273e, locked_gen: Generation(Generation(2)) } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13442025-03-25T01:31:10.484ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = BecomeUpdater
13452025-03-25T01:31:10.485ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = BecomeUpdater
13462025-03-25T01:31:10.493ZDEBGnexus (SagaExecutor): inherited lock from 5e697aaf-7503-48a7-b90c-ddad0ba0273e to cac15bc4-0f6d-49fe-bc2e-20866e2ad94c actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 5e697aaf-7503-48a7-b90c-ddad0ba0273e saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = BecomeUpdater updater_id = cac15bc4-0f6d-49fe-bc2e-20866e2ad94c
13472025-03-25T01:31:10.493ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13482025-03-25T01:31:10.493ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13492025-03-25T01:31:10.493ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("cac15bc4-0f6d-49fe-bc2e-20866e2ad94c")}) node_id = 1 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13502025-03-25T01:31:10.496ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13512025-03-25T01:31:10.496ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13522025-03-25T01:31:10.499ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13532025-03-25T01:31:10.499ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13542025-03-25T01:31:10.502ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13552025-03-25T01:31:10.502ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13562025-03-25T01:31:10.510ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13572025-03-25T01:31:10.510ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549
13582025-03-25T01:31:10.510ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UpdateNetworkConfig
13592025-03-25T01:31:10.511ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UpdateNetworkConfig
13602025-03-25T01:31:10.513ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UpdateNetworkConfig
13612025-03-25T01:31:10.513ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UpdateNetworkConfig
13622025-03-25T01:31:10.518ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
13632025-03-25T01:31:10.518ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UpdateNetworkConfig
13642025-03-25T01:31:10.518ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UpdateNetworkConfig
13652025-03-25T01:31:10.520ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
13662025-03-25T01:31:10.520ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
13672025-03-25T01:31:10.520ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
13682025-03-25T01:31:10.521ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
13692025-03-25T01:31:10.523ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
13702025-03-25T01:31:10.523ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
13712025-03-25T01:31:10.523ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
13722025-03-25T01:31:10.527ZDEBGnexus: notifying dendrite of updates instance_id = Some(7970c9b8-9b62-47f6-bc44-40934d16c549 (instance)) switch = switch0
13732025-03-25T01:31:10.528ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1574, tv_nsec: 237873542 } })
13742025-03-25T01:31:10.528ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1077
13752025-03-25T01:31:10.558ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
13762025-03-25T01:31:10.559ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13772025-03-25T01:31:10.559ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a42d4acf-f203-4542-8c9c-c2e61cd611d5", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} })
13782025-03-25T01:31:10.559ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13792025-03-25T01:31:10.560ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13802025-03-25T01:31:10.589ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
13812025-03-25T01:31:10.589ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "60a4307e-f51e-4d8a-a56d-4b0cd0b59a6b", "content-length": "4", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} })
13822025-03-25T01:31:10.591ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13832025-03-25T01:31:10.591ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13842025-03-25T01:31:10.595ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13852025-03-25T01:31:10.595ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13862025-03-25T01:31:10.602ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
13872025-03-25T01:31:10.603ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
13882025-03-25T01:31:10.620ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2025-03-25T01:31:10.608257Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, time_modified: Some(2025-03-25T01:31:10.608257Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: edb4be7f-ad8b-42c9-b534-94f7c87d25d8, time_modified: Some(2025-03-25T01:31:10.608257Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13892025-03-25T01:31:10.620ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13902025-03-25T01:31:10.620ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13912025-03-25T01:31:10.623ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
13922025-03-25T01:31:10.623ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
13932025-03-25T01:31:10.629ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13942025-03-25T01:31:10.629ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UnassignOximeterProducer
13952025-03-25T01:31:10.630ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UnassignOximeterProducer
13962025-03-25T01:31:10.634ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = a9f260fd-b911-4f87-b07b-84934591a3eb producer_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
13972025-03-25T01:31:10.634ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UnassignOximeterProducer
13982025-03-25T01:31:10.634ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = UnassignOximeterProducer
13992025-03-25T01:31:10.670ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14002025-03-25T01:31:10.670ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/7970c9b8-9b62-47f6-bc44-40934d16c549
14012025-03-25T01:31:10.699ZDEBGnexus (SagaExecutor): client response oximeter-collector = 7970c9b8-9b62-47f6-bc44-40934d16c549 result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/7970c9b8-9b62-47f6-bc44-40934d16c549", status: 204, headers: {"x-request-id": "c4c61c2e-0e98-4209-b805-a42fe6fad81b", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} }) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14022025-03-25T01:31:10.699ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = a9f260fd-b911-4f87-b07b-84934591a3eb producer_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14032025-03-25T01:31:10.702ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14042025-03-25T01:31:10.702ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14052025-03-25T01:31:10.706ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14062025-03-25T01:31:10.706ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14072025-03-25T01:31:10.710ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 lock = UpdaterLock { updater_id: cac15bc4-0f6d-49fe-bc2e-20866e2ad94c, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2025-03-25T01:31:10.456853856Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14082025-03-25T01:31:10.710ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = CommitInstanceUpdates
14092025-03-25T01:31:10.710ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = CommitInstanceUpdates
14102025-03-25T01:31:10.728ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 new_runtime = InstanceRuntimeState { time_updated: 2025-03-25T01:31:10.456853856Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14112025-03-25T01:31:10.728ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14122025-03-25T01:31:10.728ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14132025-03-25T01:31:10.728ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Signaled
14142025-03-25T01:31:10.728ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Signaled
14152025-03-25T01:31:10.728ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
14162025-03-25T01:31:10.728ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14172025-03-25T01:31:10.728ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14182025-03-25T01:31:10.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14192025-03-25T01:31:10.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14202025-03-25T01:31:10.730ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14212025-03-25T01:31:10.732ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14222025-03-25T01:31:10.732ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14232025-03-25T01:31:10.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14242025-03-25T01:31:10.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14252025-03-25T01:31:10.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14262025-03-25T01:31:10.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272025-03-25T01:31:10.735ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
14282025-03-25T01:31:10.735ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14292025-03-25T01:31:10.735ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14302025-03-25T01:31:10.735ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("7970c9b8-9b62-47f6-bc44-40934d16c549"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("1a94d6a0-c931-4ad1-aefb-f87808324f70")}) node_id = 7 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14312025-03-25T01:31:10.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14322025-03-25T01:31:10.739ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14332025-03-25T01:31:10.739ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14342025-03-25T01:31:10.741ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14352025-03-25T01:31:10.742ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14362025-03-25T01:31:10.745ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14372025-03-25T01:31:10.745ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14382025-03-25T01:31:10.748ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:82 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14392025-03-25T01:31:10.748ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ReleaseSledResources
14402025-03-25T01:31:10.748ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ReleaseSledResources
14412025-03-25T01:31:10.753ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14422025-03-25T01:31:10.753ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14432025-03-25T01:31:10.769ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
14442025-03-25T01:31:10.769ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14452025-03-25T01:31:10.770ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "138a5d74-7f75-4e13-9b83-a45711578e5c", "content-length": "372", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} })
14462025-03-25T01:31:10.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14472025-03-25T01:31:10.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14482025-03-25T01:31:10.773ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14492025-03-25T01:31:10.773ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14502025-03-25T01:31:10.777ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:113 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 propolis_id = 1a94d6a0-c931-4ad1-aefb-f87808324f70 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14512025-03-25T01:31:10.777ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = MarkVmmDeleted
14522025-03-25T01:31:10.777ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = MarkVmmDeleted
14532025-03-25T01:31:10.782ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {5ff385df-8a4d-4b05-95b1-36d92835d6d3: {10.151.1.175: {3a1bce35-9900-47e8-b587-abca91e7ba03}, 10.151.1.176: {3a1bce35-9900-47e8-b587-abca91e7ba03}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
14542025-03-25T01:31:10.782ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
14552025-03-25T01:31:10.783ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bc96e746-5d37-41cf-b3f6-671ff10a0d4c", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} })
14562025-03-25T01:31:10.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14572025-03-25T01:31:10.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14582025-03-25T01:31:10.785ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14592025-03-25T01:31:10.785ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14602025-03-25T01:31:10.785ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14612025-03-25T01:31:10.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14622025-03-25T01:31:10.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14632025-03-25T01:31:10.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14642025-03-25T01:31:10.789ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14652025-03-25T01:31:10.789ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14662025-03-25T01:31:10.791ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
14672025-03-25T01:31:10.791ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14682025-03-25T01:31:10.791ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a858e131-c9db-4f40-a06a-f3d93d9fc1bc", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} })
14692025-03-25T01:31:10.793ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 64.878189ms iteration = 25 status = {}
14702025-03-25T01:31:10.793ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14712025-03-25T01:31:10.793ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14722025-03-25T01:31:10.797ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14732025-03-25T01:31:10.797ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14742025-03-25T01:31:10.801ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14752025-03-25T01:31:10.801ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14762025-03-25T01:31:10.802ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14772025-03-25T01:31:10.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14782025-03-25T01:31:10.803ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14792025-03-25T01:31:10.803ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14802025-03-25T01:31:10.805ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14812025-03-25T01:31:10.805ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14822025-03-25T01:31:10.806ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14832025-03-25T01:31:10.809ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14842025-03-25T01:31:10.809ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, lookup_type: ById(9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7) }, key: edb4be7f-ad8b-42c9-b534-94f7c87d25d8, lookup_type: ById(edb4be7f-ad8b-42c9-b534-94f7c87d25d8) }, key: 7970c9b8-9b62-47f6-bc44-40934d16c549, lookup_type: ById(7970c9b8-9b62-47f6-bc44-40934d16c549) } result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14852025-03-25T01:31:10.809ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14862025-03-25T01:31:10.809ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_node = ChainSuccessorSaga
14872025-03-25T01:31:10.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14882025-03-25T01:31:10.811ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14892025-03-25T01:31:10.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14902025-03-25T01:31:10.813ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14912025-03-25T01:31:10.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14922025-03-25T01:31:10.814ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14932025-03-25T01:31:10.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14942025-03-25T01:31:10.816ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14952025-03-25T01:31:10.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14962025-03-25T01:31:10.817ZWARNnexus (SagaExecutor): instance update: failed to start successor saga! error = failed to fetch latest snapshot for instance file = nexus/src/app/sagas/instance_update/mod.rs:1325 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update
14972025-03-25T01:31:10.817ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
14982025-03-25T01:31:10.817ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
14992025-03-25T01:31:10.817ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Signaled
15002025-03-25T01:31:10.817ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15012025-03-25T01:31:10.817ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15022025-03-25T01:31:10.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15032025-03-25T01:31:10.819ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15042025-03-25T01:31:10.819ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15052025-03-25T01:31:10.819ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15062025-03-25T01:31:10.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15072025-03-25T01:31:10.820ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15082025-03-25T01:31:10.820ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
15092025-03-25T01:31:10.822ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15102025-03-25T01:31:10.822ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
15112025-03-25T01:31:10.822ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15122025-03-25T01:31:10.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15132025-03-25T01:31:10.825ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15142025-03-25T01:31:10.825ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
15152025-03-25T01:31:10.849ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
15162025-03-25T01:31:10.850ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cf5a057-6157-46b3-abaa-33d5f3e23a02", "content-length": "691", "date": "Tue, 25 Mar 2025 01:31:10 GMT"} })
15172025-03-25T01:31:10.850ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
15182025-03-25T01:31:10.850ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
15192025-03-25T01:31:10.850ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
15202025-03-25T01:31:10.852ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 123.579789ms iteration = 25 status = {}
15212025-03-25T01:31:10.854ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15222025-03-25T01:31:10.854ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52
15232025-03-25T01:31:10.880ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15242025-03-25T01:31:10.880ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
15252025-03-25T01:31:10.880ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 63.211325ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
15262025-03-25T01:31:10.893ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15272025-03-25T01:31:10.893ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 saga_name = instance-update saga_result = SagaResult { saga_id: b9a6f992-25a9-4190-a33e-7fa7e8d43b52, saga_log: SagaLog { saga_id: b9a6f992-25a9-4190-a33e-7fa7e8d43b52, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("cac15bc4-0f6d-49fe-bc2e-20866e2ad94c")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("cac15bc4-0f6d-49fe-bc2e-20866e2ad94c")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("7970c9b8-9b62-47f6-bc44-40934d16c549"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("1a94d6a0-c931-4ad1-aefb-f87808324f70")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("7970c9b8-9b62-47f6-bc44-40934d16c549"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("1a94d6a0-c931-4ad1-aefb-f87808324f70")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("cac15bc4-0f6d-49fe-bc2e-20866e2ad94c"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("cac15bc4-0f6d-49fe-bc2e-20866e2ad94c")}} }) }
15282025-03-25T01:31:10.893ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = b9a6f992-25a9-4190-a33e-7fa7e8d43b52 instance_id = 7970c9b8-9b62-47f6-bc44-40934d16c549 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update
15292025-03-25T01:31:10.893ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15302025-03-25T01:31:10.893ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
15312025-03-25T01:31:10.897ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15322025-03-25T01:31:10.897ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
15332025-03-25T01:31:10.901ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15342025-03-25T01:31:10.901ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
15352025-03-25T01:31:10.905ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15362025-03-25T01:31:10.905ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d
15372025-03-25T01:31:10.941ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update sec_id = 3382b509-413c-4e85-971f-4b24ca64ae0c
15382025-03-25T01:31:10.941ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = dd6b85db-a183-47a7-b825-ae8b62c5736d saga_name = start-instance-update saga_result = SagaResult { saga_id: dd6b85db-a183-47a7-b825-ae8b62c5736d, saga_log: SagaLog { saga_id: dd6b85db-a183-47a7-b825-ae8b62c5736d, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("5e697aaf-7503-48a7-b90c-ddad0ba0273e")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("5e697aaf-7503-48a7-b90c-ddad0ba0273e")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("5e697aaf-7503-48a7-b90c-ddad0ba0273e"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("5e697aaf-7503-48a7-b90c-ddad0ba0273e")}} }) }
15392025-03-25T01:31:11.010ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0a00e351-6b54-4878-a09a-7bce54f224f1 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15402025-03-25T01:31:11.010ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0a00e351-6b54-4878-a09a-7bce54f224f1 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15412025-03-25T01:31:11.012ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2342 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0a00e351-6b54-4878-a09a-7bce54f224f1 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15422025-03-25T01:31:11.741ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
15432025-03-25T01:31:11.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15442025-03-25T01:31:11.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15452025-03-25T01:31:11.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
15462025-03-25T01:31:11.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15472025-03-25T01:31:11.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15482025-03-25T01:31:11.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15492025-03-25T01:31:11.743ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15502025-03-25T01:31:11.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15512025-03-25T01:31:11.745ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.343728ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15522025-03-25T01:31:11.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15532025-03-25T01:31:11.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15542025-03-25T01:31:11.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15552025-03-25T01:31:11.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15562025-03-25T01:31:11.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15572025-03-25T01:31:11.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15582025-03-25T01:31:11.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
15592025-03-25T01:31:11.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15602025-03-25T01:31:11.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15612025-03-25T01:31:11.761ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
15622025-03-25T01:31:11.761ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.489763ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
15632025-03-25T01:31:12.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f69c4db9-aa9e-44be-80e8-6aa4056d2c31 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15642025-03-25T01:31:12.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f69c4db9-aa9e-44be-80e8-6aa4056d2c31 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15652025-03-25T01:31:12.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1612 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f69c4db9-aa9e-44be-80e8-6aa4056d2c31 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15662025-03-25T01:31:13.058ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b5c6bc8b-34ee-4112-b1f1-5e271124237f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15672025-03-25T01:31:13.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b5c6bc8b-34ee-4112-b1f1-5e271124237f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15682025-03-25T01:31:13.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2433 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b5c6bc8b-34ee-4112-b1f1-5e271124237f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15692025-03-25T01:31:14.081ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4db76a17-ec76-4557-8d7c-282cf9514f15 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15702025-03-25T01:31:14.081ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4db76a17-ec76-4557-8d7c-282cf9514f15 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15712025-03-25T01:31:14.083ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4db76a17-ec76-4557-8d7c-282cf9514f15 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15722025-03-25T01:31:15.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ca8e62e9-1828-4ffe-a5f0-1e164da3652a roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15732025-03-25T01:31:15.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ca8e62e9-1828-4ffe-a5f0-1e164da3652a resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15742025-03-25T01:31:15.106ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2365 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ca8e62e9-1828-4ffe-a5f0-1e164da3652a response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15752025-03-25T01:31:15.760ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:40188
15762025-03-25T01:31:15.760ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40188 req_id = 52f08c58-8649-4ec9-ba66-d31c5b51a513 roles = RoleSet { roles: {} } uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?sort_by=id_ascending
15772025-03-25T01:31:15.760ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40188 req_id = 52f08c58-8649-4ec9-ba66-d31c5b51a513 resource = Database result = Ok(()) uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?sort_by=id_ascending
15782025-03-25T01:31:15.763ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3325 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40188 req_id = 52f08c58-8649-4ec9-ba66-d31c5b51a513 response_code = 200 uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?sort_by=id_ascending
15792025-03-25T01:31:15.764ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40188 req_id = fa1175f5-0028-4ded-9884-9586141719d3 roles = RoleSet { roles: {} } uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4MzI5NzcxOC01ZjQyLTQ3NmMtYTA2Ni1iODU2ZmRlODVmMjUifX0%3D
15802025-03-25T01:31:15.764ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40188 req_id = fa1175f5-0028-4ded-9884-9586141719d3 resource = Database result = Ok(()) uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4MzI5NzcxOC01ZjQyLTQ3NmMtYTA2Ni1iODU2ZmRlODVmMjUifX0%3D
15812025-03-25T01:31:15.766ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2721 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:40188 req_id = fa1175f5-0028-4ded-9884-9586141719d3 response_code = 200 uri = /metrics/collectors/a9f260fd-b911-4f87-b07b-84934591a3eb/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4MzI5NzcxOC01ZjQyLTQ3NmMtYTA2Ni1iODU2ZmRlODVmMjUifX0%3D
15822025-03-25T01:31:15.768ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 529 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 415693bd-cff5-4d24-bd38-30b55e8851db response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
15832025-03-25T01:31:16.127ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f3d9824e-3fa4-4093-a7a1-5d094f33365b roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15842025-03-25T01:31:16.127ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f3d9824e-3fa4-4093-a7a1-5d094f33365b resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15852025-03-25T01:31:16.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2548 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f3d9824e-3fa4-4093-a7a1-5d094f33365b response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15862025-03-25T01:31:17.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b511077b-1132-4ef9-a20c-930f82059aa7 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15872025-03-25T01:31:17.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b511077b-1132-4ef9-a20c-930f82059aa7 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15882025-03-25T01:31:17.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2735 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b511077b-1132-4ef9-a20c-930f82059aa7 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15892025-03-25T01:31:18.174ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 29f886de-9c40-4387-bf11-2948e8e9c54f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15902025-03-25T01:31:18.175ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 29f886de-9c40-4387-bf11-2948e8e9c54f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15912025-03-25T01:31:18.177ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2345 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 29f886de-9c40-4387-bf11-2948e8e9c54f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15922025-03-25T01:31:19.197ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2bba5300-7b96-4805-b23b-f237883b6798 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15932025-03-25T01:31:19.198ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2bba5300-7b96-4805-b23b-f237883b6798 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15942025-03-25T01:31:19.200ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2958 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2bba5300-7b96-4805-b23b-f237883b6798 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15952025-03-25T01:31:20.220ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 022e3b2f-2f91-489b-af48-63073f9150c1 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15962025-03-25T01:31:20.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 022e3b2f-2f91-489b-af48-63073f9150c1 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15972025-03-25T01:31:20.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2021 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 022e3b2f-2f91-489b-af48-63073f9150c1 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15982025-03-25T01:31:21.244ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 359d2253-e71d-40cc-8e67-3e8590f67383 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
15992025-03-25T01:31:21.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 359d2253-e71d-40cc-8e67-3e8590f67383 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16002025-03-25T01:31:21.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2537 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 359d2253-e71d-40cc-8e67-3e8590f67383 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16012025-03-25T01:31:21.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
16022025-03-25T01:31:21.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16032025-03-25T01:31:21.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16042025-03-25T01:31:21.742ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
16052025-03-25T01:31:21.742ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16062025-03-25T01:31:21.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16072025-03-25T01:31:21.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16082025-03-25T01:31:21.744ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16092025-03-25T01:31:21.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16102025-03-25T01:31:21.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16112025-03-25T01:31:21.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16122025-03-25T01:31:21.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16132025-03-25T01:31:21.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16142025-03-25T01:31:21.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16152025-03-25T01:31:21.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16162025-03-25T01:31:21.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
16172025-03-25T01:31:21.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16182025-03-25T01:31:21.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16192025-03-25T01:31:21.766ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
16202025-03-25T01:31:21.766ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.296702ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
16212025-03-25T01:31:21.776ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.27293ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16222025-03-25T01:31:22.267ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ffb33511-b8fd-4b9c-b9e9-1af52044b99b roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16232025-03-25T01:31:22.268ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ffb33511-b8fd-4b9c-b9e9-1af52044b99b resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16242025-03-25T01:31:22.270ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ffb33511-b8fd-4b9c-b9e9-1af52044b99b response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16252025-03-25T01:31:23.289ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7348f18e-d335-4f61-8cfb-2b501a0d5d93 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16262025-03-25T01:31:23.290ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7348f18e-d335-4f61-8cfb-2b501a0d5d93 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16272025-03-25T01:31:23.292ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2410 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7348f18e-d335-4f61-8cfb-2b501a0d5d93 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16282025-03-25T01:31:24.311ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cf0ab0a1-347c-437b-9d49-443e7f83f6e3 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16292025-03-25T01:31:24.312ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cf0ab0a1-347c-437b-9d49-443e7f83f6e3 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16302025-03-25T01:31:24.313ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1991 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cf0ab0a1-347c-437b-9d49-443e7f83f6e3 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16312025-03-25T01:31:25.334ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 417a6722-bb48-4626-970b-c119ad463088 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16322025-03-25T01:31:25.334ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 417a6722-bb48-4626-970b-c119ad463088 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16332025-03-25T01:31:25.337ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2916 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 417a6722-bb48-4626-970b-c119ad463088 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16342025-03-25T01:31:25.768ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 516 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = f4745040-e4cb-421d-a6f9-ff6e2b6c8cf7 response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
16352025-03-25T01:31:26.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 65bc6c8c-7d26-4e98-8736-e6bc09fa46c2 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16362025-03-25T01:31:26.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 65bc6c8c-7d26-4e98-8736-e6bc09fa46c2 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16372025-03-25T01:31:26.359ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2699 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 65bc6c8c-7d26-4e98-8736-e6bc09fa46c2 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16382025-03-25T01:31:27.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7cd4862c-eb89-4e6d-b02b-79222bb50607 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16392025-03-25T01:31:27.381ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7cd4862c-eb89-4e6d-b02b-79222bb50607 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16402025-03-25T01:31:27.383ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2697 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 7cd4862c-eb89-4e6d-b02b-79222bb50607 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16412025-03-25T01:31:28.403ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9e1279c2-bb22-43db-897a-fab2b189ed2e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16422025-03-25T01:31:28.403ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9e1279c2-bb22-43db-897a-fab2b189ed2e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16432025-03-25T01:31:28.405ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9e1279c2-bb22-43db-897a-fab2b189ed2e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16442025-03-25T01:31:29.425ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cbdaeecc-995c-4819-99a1-2639c12bfbd8 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16452025-03-25T01:31:29.426ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cbdaeecc-995c-4819-99a1-2639c12bfbd8 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16462025-03-25T01:31:29.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cbdaeecc-995c-4819-99a1-2639c12bfbd8 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16472025-03-25T01:31:30.448ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcb96423-fe24-4ad9-bb3a-2531c7ca4171 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16482025-03-25T01:31:30.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcb96423-fe24-4ad9-bb3a-2531c7ca4171 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16492025-03-25T01:31:30.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2255 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = dcb96423-fe24-4ad9-bb3a-2531c7ca4171 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16502025-03-25T01:31:31.471ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 547fb96f-7517-465d-bc46-7095574bd8f5 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16512025-03-25T01:31:31.471ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 547fb96f-7517-465d-bc46-7095574bd8f5 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16522025-03-25T01:31:31.473ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2376 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 547fb96f-7517-465d-bc46-7095574bd8f5 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
16532025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
16542025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
16552025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16562025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
16572025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16582025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
16592025-03-25T01:31:31.741ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
16602025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
16612025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16622025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
16632025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16642025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
16652025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
16662025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
16672025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16682025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16692025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16702025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
16712025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
16722025-03-25T01:31:31.741ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-03-25 01:21:31.741481144 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
16732025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
16742025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
16752025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
16762025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 25 reason = Timeout
16772025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16782025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
16792025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16802025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16812025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
16822025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
16832025-03-25T01:31:31.741ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = blueprint disabled target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b
16842025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
16852025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16862025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 21.32\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b"}
16872025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
16882025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
16892025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16902025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16912025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
16922025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16932025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
16942025-03-25T01:31:31.741ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
16952025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
16962025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16972025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16982025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
16992025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
17002025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17012025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
17022025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
17032025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
17042025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17052025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
17062025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17072025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17082025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17092025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17102025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
17112025-03-25T01:31:31.742ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
17122025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
17132025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
17142025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
17152025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17162025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17172025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
17182025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
17192025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
17202025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
17212025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17222025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
17232025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
17242025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
17252025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17262025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
17272025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
17282025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17292025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
17302025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17312025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17322025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
17332025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17342025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17352025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
17362025-03-25T01:31:31.742ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("3f0307c3-79bb-4dd8-87b6-ae6655144209.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("40054b36-2618-459b-aad0-799de8938fb6.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("b6a22759-03b9-4dff-b9a0-d413b6908c97.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("b6a22759-03b9-4dff-b9a0-d413b6908c97.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:2::1))) }], valid_until: Instant { tv_sec: 1595, tv_nsec: 451606380 } })
17372025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
17382025-03-25T01:31:31.742ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("4b6e96f6-cd23-4897-9bfb-96b629ccf872.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("658f8989-b24e-48cb-83e6-b3a3b15d47d4.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("658f8989-b24e-48cb-83e6-b3a3b15d47d4.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::9))) }], valid_until: Instant { tv_sec: 1595, tv_nsec: 451621380 } })
17392025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17402025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17412025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17422025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
17432025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
17442025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
17452025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
17462025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
17472025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17482025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17492025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
17502025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17512025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17522025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17532025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17542025-03-25T01:31:31.742ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17552025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17562025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
17572025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
17582025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.764715ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
17592025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17602025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.685355ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
17612025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17622025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17632025-03-25T01:31:31.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17642025-03-25T01:31:31.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17652025-03-25T01:31:31.744ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17662025-03-25T01:31:31.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17672025-03-25T01:31:31.745ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.120779ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
17682025-03-25T01:31:31.745ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17692025-03-25T01:31:31.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17702025-03-25T01:31:31.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17712025-03-25T01:31:31.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17722025-03-25T01:31:31.747ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17732025-03-25T01:31:31.747ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17742025-03-25T01:31:31.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17752025-03-25T01:31:31.748ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17762025-03-25T01:31:31.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17772025-03-25T01:31:31.748ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-03-25 01:21:31.741481144 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
17782025-03-25T01:31:31.748ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.442566ms iteration = 13 status = {"expiration":"2025-03-25T01:21:31.741481144Z","pruned":[]}
17792025-03-25T01:31:31.750ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1595, tv_nsec: 459796997 } })
17802025-03-25T01:31:31.750ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17812025-03-25T01:31:31.750ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
17822025-03-25T01:31:31.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17832025-03-25T01:31:31.751ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17842025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17852025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17862025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17872025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17882025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17892025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17902025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
17912025-03-25T01:31:31.754ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
17922025-03-25T01:31:31.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
17932025-03-25T01:31:31.756ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 14.851373ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
17942025-03-25T01:31:31.757ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17952025-03-25T01:31:31.757ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
17962025-03-25T01:31:31.757ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17972025-03-25T01:31:31.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17982025-03-25T01:31:31.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17992025-03-25T01:31:31.758ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002025-03-25T01:31:31.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18012025-03-25T01:31:31.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18022025-03-25T01:31:31.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18032025-03-25T01:31:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18042025-03-25T01:31:31.765ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18052025-03-25T01:31:31.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
18062025-03-25T01:31:31.768ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
18072025-03-25T01:31:31.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
18082025-03-25T01:31:31.769ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1595, tv_nsec: 478551755 } })
18092025-03-25T01:31:31.769ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
18102025-03-25T01:31:31.770ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18112025-03-25T01:31:31.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18122025-03-25T01:31:31.773ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18132025-03-25T01:31:31.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18142025-03-25T01:31:31.773ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18152025-03-25T01:31:31.773ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18162025-03-25T01:31:31.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18172025-03-25T01:31:31.776ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18182025-03-25T01:31:31.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18192025-03-25T01:31:31.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18202025-03-25T01:31:31.781ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18212025-03-25T01:31:31.794ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18222025-03-25T01:31:31.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 53.209611ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
18232025-03-25T01:31:31.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18242025-03-25T01:31:31.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18252025-03-25T01:31:31.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18262025-03-25T01:31:31.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18272025-03-25T01:31:31.802ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18282025-03-25T01:31:31.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18292025-03-25T01:31:31.803ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1595, tv_nsec: 512661663 } })
18302025-03-25T01:31:31.803ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
18312025-03-25T01:31:31.806ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18322025-03-25T01:31:31.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
18332025-03-25T01:31:31.810ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
18342025-03-25T01:31:31.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
18352025-03-25T01:31:31.812ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18362025-03-25T01:31:31.814ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18372025-03-25T01:31:31.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18382025-03-25T01:31:31.815ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18392025-03-25T01:31:31.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18402025-03-25T01:31:31.817ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
18412025-03-25T01:31:31.817ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
18422025-03-25T01:31:31.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
18432025-03-25T01:31:31.830ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
18442025-03-25T01:31:31.830ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:167 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
18452025-03-25T01:31:31.830ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 88.203932ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
18462025-03-25T01:31:31.840ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18472025-03-25T01:31:31.840ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 98.718623ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
18482025-03-25T01:31:31.841ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18492025-03-25T01:31:31.841ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18502025-03-25T01:31:31.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18512025-03-25T01:31:31.842ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 100.638791ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
18522025-03-25T01:31:31.844ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
18532025-03-25T01:31:31.844ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
18542025-03-25T01:31:31.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
18552025-03-25T01:31:31.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18562025-03-25T01:31:31.845ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18572025-03-25T01:31:31.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18582025-03-25T01:31:31.848ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18592025-03-25T01:31:31.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18602025-03-25T01:31:31.850ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18612025-03-25T01:31:31.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18622025-03-25T01:31:31.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18632025-03-25T01:31:31.852ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18642025-03-25T01:31:31.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18652025-03-25T01:31:31.852ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18662025-03-25T01:31:31.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
18672025-03-25T01:31:31.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18682025-03-25T01:31:31.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18692025-03-25T01:31:31.855ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
18702025-03-25T01:31:31.855ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
18712025-03-25T01:31:31.855ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18722025-03-25T01:31:31.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
18732025-03-25T01:31:31.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18742025-03-25T01:31:31.856ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 114.091783ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
18752025-03-25T01:31:31.857ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18762025-03-25T01:31:31.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18772025-03-25T01:31:31.858ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
18782025-03-25T01:31:31.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
18792025-03-25T01:31:31.859ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
18802025-03-25T01:31:31.859ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
18812025-03-25T01:31:31.859ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
18822025-03-25T01:31:31.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18832025-03-25T01:31:31.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18842025-03-25T01:31:31.860ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 118.664798ms iteration = 6 status = {"nsuccess":5}
18852025-03-25T01:31:31.860ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18862025-03-25T01:31:31.863ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18872025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
18882025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
18892025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
18902025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18912025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18922025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18932025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18942025-03-25T01:31:31.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18952025-03-25T01:31:31.866ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
18962025-03-25T01:31:31.866ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 124.616444ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
18972025-03-25T01:31:31.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18982025-03-25T01:31:31.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18992025-03-25T01:31:31.868ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19002025-03-25T01:31:31.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19012025-03-25T01:31:31.869ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19022025-03-25T01:31:31.869ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
19032025-03-25T01:31:31.869ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
19042025-03-25T01:31:31.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
19052025-03-25T01:31:31.869ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 127.859439ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19062025-03-25T01:31:31.870ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
19072025-03-25T01:31:31.870ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 129.054487ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
19082025-03-25T01:31:31.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19092025-03-25T01:31:31.872ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19102025-03-25T01:31:31.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19112025-03-25T01:31:31.875ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
19122025-03-25T01:31:31.875ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
19132025-03-25T01:31:31.875ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 133.952306ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
19142025-03-25T01:31:31.880ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19152025-03-25T01:31:31.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19162025-03-25T01:31:31.882ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19172025-03-25T01:31:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19182025-03-25T01:31:31.886ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19192025-03-25T01:31:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19202025-03-25T01:31:31.890ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
19212025-03-25T01:31:31.894ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19222025-03-25T01:31:31.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19232025-03-25T01:31:31.897ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19242025-03-25T01:31:31.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19252025-03-25T01:31:31.898ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
19262025-03-25T01:31:31.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
19272025-03-25T01:31:31.902ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19282025-03-25T01:31:31.905ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 163.106324ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
19292025-03-25T01:31:31.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 1768bc4c-c9ec-4fc7-a560-5ef338d49e99, lookup_type: ById(1768bc4c-c9ec-4fc7-a560-5ef338d49e99) } result = Ok(())
19302025-03-25T01:31:31.906ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19312025-03-25T01:31:31.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19322025-03-25T01:31:31.911ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 169.735929ms iteration = 25 status = 0
19332025-03-25T01:31:31.923ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
19342025-03-25T01:31:31.926ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19352025-03-25T01:31:31.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19362025-03-25T01:31:31.930ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 188.181153ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
19372025-03-25T01:31:31.930ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19382025-03-25T01:31:31.932ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19392025-03-25T01:31:31.932ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 1e54f7e5-0241-40be-bb43-5fe289855c17 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19402025-03-25T01:31:31.932ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19412025-03-25T01:31:31.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19422025-03-25T01:31:31.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19432025-03-25T01:31:31.933ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19442025-03-25T01:31:31.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19452025-03-25T01:31:31.935ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
19462025-03-25T01:31:31.935ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
19472025-03-25T01:31:31.935ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 193.859254ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
19482025-03-25T01:31:31.936ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19492025-03-25T01:31:31.938ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd zone_address = fd00:1122:3344:101::2
19502025-03-25T01:31:31.938ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19512025-03-25T01:31:31.939ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49359487-c6a5-483d-a81a-0c122ac506fc", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
19522025-03-25T01:31:31.939ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19532025-03-25T01:31:31.939ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
19542025-03-25T01:31:31.939ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 60210834-8447-4faf-a5df-01a572eb2d5e dns_zone_name = oxide.test found_so_far = 0 version = 2
19552025-03-25T01:31:31.939ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19562025-03-25T01:31:31.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19572025-03-25T01:31:31.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19582025-03-25T01:31:31.940ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19592025-03-25T01:31:31.941ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
19602025-03-25T01:31:31.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19612025-03-25T01:31:31.941ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19622025-03-25T01:31:31.943ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 1e54f7e5-0241-40be-bb43-5fe289855c17 dns_zone_name = control-plane.oxide.internal found_so_far = 52 version = 1
19632025-03-25T01:31:31.943ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
19642025-03-25T01:31:31.943ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-03-25 01:20:06.503139 UTC dns_group = internal generation = 1
19652025-03-25T01:31:31.943ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 202.389626ms iteration = 13 status = {"generation":1}
19662025-03-25T01:31:31.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19672025-03-25T01:31:31.946ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19682025-03-25T01:31:31.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19692025-03-25T01:31:31.947ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 205.687443ms iteration = 28 status = {"physical_disks_added":0}
19702025-03-25T01:31:31.948ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 60210834-8447-4faf-a5df-01a572eb2d5e dns_zone_name = oxide.test found_so_far = 1 version = 2
19712025-03-25T01:31:31.948ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
19722025-03-25T01:31:31.948ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-03-25 01:20:07.107344 UTC dns_group = external generation = 2
19732025-03-25T01:31:31.948ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 207.182705ms iteration = 13 status = {"generation":2}
19742025-03-25T01:31:31.948ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
19752025-03-25T01:31:31.948ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 207.390877ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
19762025-03-25T01:31:31.984ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-03-25T01:19:30.034299Z, time_modified: 2025-03-25T01:19:30.034299Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, db_silo: Silo { identity: SiloIdentity { id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-25T01:20:07.062083Z, time_modified: 2025-03-25T01:20:07.062083Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, db_silo: Silo { identity: SiloIdentity { id: 9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-03-25T01:20:07.062083Z, time_modified: 2025-03-25T01:20:07.062083Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
19772025-03-25T01:31:31.984ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 243.476994ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7","tls_certs":["e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175"]}},"default_endpoint":{"silo_id":"9a81b8ef-a36f-4a8b-b795-0c9d1428d3c7","tls_certs":["e09544000d777870bebf1c9c16600f284e556613366f71437328220a360af175"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
19782025-03-25T01:31:31.988ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19792025-03-25T01:31:31.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19802025-03-25T01:31:31.990ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19812025-03-25T01:31:31.992ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19822025-03-25T01:31:31.993ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19832025-03-25T01:31:31.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19842025-03-25T01:31:31.995ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19852025-03-25T01:31:31.997ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19862025-03-25T01:31:31.997ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19872025-03-25T01:31:31.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19882025-03-25T01:31:32.001ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
19892025-03-25T01:31:32.001ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19902025-03-25T01:31:32.002ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "01621aee-e81c-4282-80c3-cbe20c833ef4", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
19912025-03-25T01:31:32.002ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19922025-03-25T01:31:32.005ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19932025-03-25T01:31:32.042ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
19942025-03-25T01:31:32.042ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 300.399059ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
19952025-03-25T01:31:32.094ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
19962025-03-25T01:31:32.094ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19972025-03-25T01:31:32.096ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ced2dba-d3ab-482f-8a7a-b71116326703", "content-length": "26", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
19982025-03-25T01:31:32.096ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19992025-03-25T01:31:32.097ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
20002025-03-25T01:31:32.098ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
20012025-03-25T01:31:32.123ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
20022025-03-25T01:31:32.123ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "840455f5-675c-4483-b695-7804bfb8c477", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20032025-03-25T01:31:32.123ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20042025-03-25T01:31:32.125ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7758a222-e885-4e15-8b17-8546aaad790d", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20052025-03-25T01:31:32.125ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20062025-03-25T01:31:32.127ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
20072025-03-25T01:31:32.127ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 385.695899ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
20082025-03-25T01:31:32.194ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
20092025-03-25T01:31:32.194ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "87542c69-01c7-4868-a4ba-61c1e22b5cc4", "content-length": "691", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} })
20102025-03-25T01:31:32.194ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
20112025-03-25T01:31:32.194ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
20122025-03-25T01:31:32.194ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
20132025-03-25T01:31:32.196ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 454.413052ms iteration = 26 status = {}
20142025-03-25T01:31:32.203ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
20152025-03-25T01:31:32.203ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cdd86df-924b-4028-a94b-478f7e22086c", "content-length": "2", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
20162025-03-25T01:31:32.205ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
20172025-03-25T01:31:32.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
20182025-03-25T01:31:32.208ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 466.959262ms iteration = 25 status = null
20192025-03-25T01:31:32.252ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
20202025-03-25T01:31:32.252ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "9ea3b4e5-a6ed-49aa-b6a2-3b905c36aee5", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20212025-03-25T01:31:32.252ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20222025-03-25T01:31:32.255ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "50c960ba-ef8a-4f32-a02a-a45e96bad0e7", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20232025-03-25T01:31:32.255ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20242025-03-25T01:31:32.256ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "23f1ff59-9816-4790-9cc3-bbaf82197c94", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20252025-03-25T01:31:32.256ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20262025-03-25T01:31:32.258ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
20272025-03-25T01:31:32.258ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 516.847775ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
20282025-03-25T01:31:32.296ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
20292025-03-25T01:31:32.296ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20302025-03-25T01:31:32.296ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "554260d4-46a6-46d8-ba6a-899fbfd11295", "content-length": "372", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} })
20312025-03-25T01:31:32.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20322025-03-25T01:31:32.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20332025-03-25T01:31:32.306ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 565.031144ms iteration = 25 status = {}
20342025-03-25T01:31:32.309ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
20352025-03-25T01:31:32.309ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
20362025-03-25T01:31:32.309ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bbee1b59-af39-4fed-9aa7-d927c8830255", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} })
20372025-03-25T01:31:32.309ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20382025-03-25T01:31:32.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20392025-03-25T01:31:32.312ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20402025-03-25T01:31:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20412025-03-25T01:31:32.314ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20422025-03-25T01:31:32.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20432025-03-25T01:31:32.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20442025-03-25T01:31:32.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20452025-03-25T01:31:32.317ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1a5160a0-1d63-486b-827e-4b7de2ad2079, rack_id: 16a011db-af85-49cc-8fa6-b468f46a78bd, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(80c078c2-e0e7-4e11-a9de-db9b8488eb2c) } rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
20462025-03-25T01:31:32.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20472025-03-25T01:31:32.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20482025-03-25T01:31:32.317ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
20492025-03-25T01:31:32.317ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
20502025-03-25T01:31:32.318ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f9957dbf-36c3-4edd-890c-75a62c006094", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} })
20512025-03-25T01:31:32.320ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 578.442617ms iteration = 26 status = {}
20522025-03-25T01:31:32.325ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
20532025-03-25T01:31:32.326ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e01c45a8-c927-4c74-88d7-650b34a5a568", "content-length": "134", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
20542025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
20552025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 80c078c2-e0e7-4e11-a9de-db9b8488eb2c
20562025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
20572025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
20582025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20592025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20602025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20612025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20622025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
20632025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
20642025-03-25T01:31:32.326ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
20652025-03-25T01:31:32.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20662025-03-25T01:31:32.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20672025-03-25T01:31:32.329ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
20682025-03-25T01:31:32.329ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "a44cb77d-e294-4e60-a525-4e0f026a03e6", "content-length": "2", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
20692025-03-25T01:31:32.329ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
20702025-03-25T01:31:32.329ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6232bf8-e9c9-444e-bae2-73ed001496b1", "content-length": "51", "date": "Tue, 25 Mar 2025 01:31:31 GMT"} })
20712025-03-25T01:31:32.329ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
20722025-03-25T01:31:32.329ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
20732025-03-25T01:31:32.424ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c01a7129-b8a0-443a-a9cd-cfede8cf1c6d", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} })
20742025-03-25T01:31:32.424ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch = Switch0
20752025-03-25T01:31:32.424ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20762025-03-25T01:31:32.424ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "63d38001-fcf5-4c19-a2ac-61c69d471c93", "content-length": "682", "date": "Tue, 25 Mar 2025 01:31:32 GMT"} })
20772025-03-25T01:31:32.424ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20782025-03-25T01:31:32.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20792025-03-25T01:31:32.428ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20802025-03-25T01:31:32.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20812025-03-25T01:31:32.430ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20822025-03-25T01:31:32.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20832025-03-25T01:31:32.433ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20842025-03-25T01:31:32.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20852025-03-25T01:31:32.444ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 702.764705ms iteration = 25 status = {}
20862025-03-25T01:31:32.494ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 024883bb-4a59-49ee-a11f-a3ef116f3e8e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20872025-03-25T01:31:32.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 024883bb-4a59-49ee-a11f-a3ef116f3e8e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20882025-03-25T01:31:32.496ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1997 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 024883bb-4a59-49ee-a11f-a3ef116f3e8e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20892025-03-25T01:31:33.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = baadbba0-b726-4d4a-8885-cf5f039fce99 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20902025-03-25T01:31:33.520ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = baadbba0-b726-4d4a-8885-cf5f039fce99 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20912025-03-25T01:31:33.521ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2101 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = baadbba0-b726-4d4a-8885-cf5f039fce99 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20922025-03-25T01:31:34.542ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 034e0236-678c-4e62-bc9f-1b786c8b2663 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20932025-03-25T01:31:34.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 034e0236-678c-4e62-bc9f-1b786c8b2663 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20942025-03-25T01:31:34.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 034e0236-678c-4e62-bc9f-1b786c8b2663 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20952025-03-25T01:31:35.565ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b5aad71a-f774-45e4-801f-8c73370a1bc0 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20962025-03-25T01:31:35.566ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b5aad71a-f774-45e4-801f-8c73370a1bc0 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20972025-03-25T01:31:35.568ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2667 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = b5aad71a-f774-45e4-801f-8c73370a1bc0 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
20982025-03-25T01:31:35.768ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 781 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = daecb80d-ac1e-4a56-aeff-854abed8f34d response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
20992025-03-25T01:31:36.588ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4bda41ef-dfeb-4e63-8c9b-6845b704fdfa roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21002025-03-25T01:31:36.588ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4bda41ef-dfeb-4e63-8c9b-6845b704fdfa resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21012025-03-25T01:31:36.590ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2500 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4bda41ef-dfeb-4e63-8c9b-6845b704fdfa response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21022025-03-25T01:31:37.610ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 160ddb0d-152f-4ecb-ad9d-13ebf9cfd08e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21032025-03-25T01:31:37.610ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 160ddb0d-152f-4ecb-ad9d-13ebf9cfd08e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21042025-03-25T01:31:37.612ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1973 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 160ddb0d-152f-4ecb-ad9d-13ebf9cfd08e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21052025-03-25T01:31:38.632ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e40119ec-17ad-44a9-8a95-becc35f08a82 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21062025-03-25T01:31:38.632ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e40119ec-17ad-44a9-8a95-becc35f08a82 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21072025-03-25T01:31:38.634ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2530 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e40119ec-17ad-44a9-8a95-becc35f08a82 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21082025-03-25T01:31:39.654ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 3cb685b3-30ad-4b3a-b788-a5e94d3e4db3 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21092025-03-25T01:31:39.654ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 3cb685b3-30ad-4b3a-b788-a5e94d3e4db3 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21102025-03-25T01:31:39.656ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2143 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 3cb685b3-30ad-4b3a-b788-a5e94d3e4db3 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21112025-03-25T01:31:40.676ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = bfcd8d89-4125-4d08-b1b2-65e977e8dd51 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21122025-03-25T01:31:40.676ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = bfcd8d89-4125-4d08-b1b2-65e977e8dd51 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21132025-03-25T01:31:40.678ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2694 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = bfcd8d89-4125-4d08-b1b2-65e977e8dd51 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21142025-03-25T01:31:41.699ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = eaa2ae1b-5bcf-454c-9817-4633470a29cd roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21152025-03-25T01:31:41.699ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = eaa2ae1b-5bcf-454c-9817-4633470a29cd resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21162025-03-25T01:31:41.701ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = eaa2ae1b-5bcf-454c-9817-4633470a29cd response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21172025-03-25T01:31:41.742ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
21182025-03-25T01:31:41.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21192025-03-25T01:31:41.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21202025-03-25T01:31:41.742ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
21212025-03-25T01:31:41.742ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21222025-03-25T01:31:41.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21232025-03-25T01:31:41.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21242025-03-25T01:31:41.744ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21252025-03-25T01:31:41.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21262025-03-25T01:31:41.746ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.420029ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21272025-03-25T01:31:41.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21282025-03-25T01:31:41.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21292025-03-25T01:31:41.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21302025-03-25T01:31:41.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21312025-03-25T01:31:41.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21322025-03-25T01:31:41.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21332025-03-25T01:31:41.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
21342025-03-25T01:31:41.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21352025-03-25T01:31:41.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21362025-03-25T01:31:41.758ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
21372025-03-25T01:31:41.758ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 16.309883ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
21382025-03-25T01:31:42.722ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 32538e40-6562-4aa5-b5cb-f6b78ec86e1a roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21392025-03-25T01:31:42.723ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 32538e40-6562-4aa5-b5cb-f6b78ec86e1a resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21402025-03-25T01:31:42.724ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2314 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 32538e40-6562-4aa5-b5cb-f6b78ec86e1a response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21412025-03-25T01:31:43.745ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6d952fc5-1459-4a1a-96c6-31cc86063d64 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21422025-03-25T01:31:43.745ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6d952fc5-1459-4a1a-96c6-31cc86063d64 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21432025-03-25T01:31:43.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1948 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 6d952fc5-1459-4a1a-96c6-31cc86063d64 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21442025-03-25T01:31:44.767ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 167c59f9-8802-462d-8d3d-8394c9e9d00c roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21452025-03-25T01:31:44.768ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 167c59f9-8802-462d-8d3d-8394c9e9d00c resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21462025-03-25T01:31:44.770ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2408 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 167c59f9-8802-462d-8d3d-8394c9e9d00c response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21472025-03-25T01:31:45.768ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 470 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 1ab6fb0c-5b83-480c-a00a-a25a4607ae8b response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
21482025-03-25T01:31:45.791ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f661e09c-6d16-4749-bc55-83aa59410710 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21492025-03-25T01:31:45.791ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f661e09c-6d16-4749-bc55-83aa59410710 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21502025-03-25T01:31:45.793ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2266 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f661e09c-6d16-4749-bc55-83aa59410710 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21512025-03-25T01:31:46.814ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c9a0edb6-122e-4291-aaf9-bfeba1897990 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21522025-03-25T01:31:46.814ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c9a0edb6-122e-4291-aaf9-bfeba1897990 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21532025-03-25T01:31:46.815ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1814 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c9a0edb6-122e-4291-aaf9-bfeba1897990 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21542025-03-25T01:31:47.835ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ab9e8a7c-1c8a-4de1-84bb-2d3017928fd3 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21552025-03-25T01:31:47.835ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ab9e8a7c-1c8a-4de1-84bb-2d3017928fd3 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21562025-03-25T01:31:47.837ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2558 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = ab9e8a7c-1c8a-4de1-84bb-2d3017928fd3 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21572025-03-25T01:31:48.858ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = a63e032a-4d4e-4623-be46-90f5468f32d6 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21582025-03-25T01:31:48.858ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = a63e032a-4d4e-4623-be46-90f5468f32d6 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21592025-03-25T01:31:48.860ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2261 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = a63e032a-4d4e-4623-be46-90f5468f32d6 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21602025-03-25T01:31:49.880ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 8591cdda-5b74-4a97-82d6-1cfdd6944e49 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21612025-03-25T01:31:49.880ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 8591cdda-5b74-4a97-82d6-1cfdd6944e49 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21622025-03-25T01:31:49.882ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2378 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 8591cdda-5b74-4a97-82d6-1cfdd6944e49 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21632025-03-25T01:31:50.903ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0e0cd634-5358-427f-8fbd-ec535f4a41cf roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21642025-03-25T01:31:50.903ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0e0cd634-5358-427f-8fbd-ec535f4a41cf resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21652025-03-25T01:31:50.905ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2157 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 0e0cd634-5358-427f-8fbd-ec535f4a41cf response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21662025-03-25T01:31:51.741ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
21672025-03-25T01:31:51.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21682025-03-25T01:31:51.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21692025-03-25T01:31:51.742ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
21702025-03-25T01:31:51.742ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21712025-03-25T01:31:51.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21722025-03-25T01:31:51.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21732025-03-25T01:31:51.744ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21742025-03-25T01:31:51.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21752025-03-25T01:31:51.746ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.279221ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21762025-03-25T01:31:51.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21772025-03-25T01:31:51.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21782025-03-25T01:31:51.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21792025-03-25T01:31:51.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21802025-03-25T01:31:51.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21812025-03-25T01:31:51.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21822025-03-25T01:31:51.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
21832025-03-25T01:31:51.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21842025-03-25T01:31:51.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21852025-03-25T01:31:51.758ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
21862025-03-25T01:31:51.759ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.353008ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
21872025-03-25T01:31:51.924ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e7d1321a-6853-4c71-92f1-ffe16eaab9da roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21882025-03-25T01:31:51.924ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e7d1321a-6853-4c71-92f1-ffe16eaab9da resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21892025-03-25T01:31:51.926ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2161 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = e7d1321a-6853-4c71-92f1-ffe16eaab9da response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21902025-03-25T01:31:52.946ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 20b61727-5152-4df3-b94c-0fc8a89b9a5e roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21912025-03-25T01:31:52.947ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 20b61727-5152-4df3-b94c-0fc8a89b9a5e resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21922025-03-25T01:31:52.949ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2445 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 20b61727-5152-4df3-b94c-0fc8a89b9a5e response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21932025-03-25T01:31:53.969ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9aaf5bff-866b-446a-8579-9f6796f23136 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21942025-03-25T01:31:53.970ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9aaf5bff-866b-446a-8579-9f6796f23136 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21952025-03-25T01:31:53.971ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 9aaf5bff-866b-446a-8579-9f6796f23136 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21962025-03-25T01:31:54.992ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4fc75f08-a362-41e5-b31e-6247598f0b0d roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21972025-03-25T01:31:54.992ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4fc75f08-a362-41e5-b31e-6247598f0b0d resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21982025-03-25T01:31:54.994ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3001 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 4fc75f08-a362-41e5-b31e-6247598f0b0d response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
21992025-03-25T01:31:55.768ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 552 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 287331e6-da51-4c5f-a478-b1a6854fdc4d response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
22002025-03-25T01:31:56.019ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 657a2a37-c1bf-4970-acad-00d3dcac7f45 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22012025-03-25T01:31:56.019ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 657a2a37-c1bf-4970-acad-00d3dcac7f45 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22022025-03-25T01:31:56.021ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1938 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 657a2a37-c1bf-4970-acad-00d3dcac7f45 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22032025-03-25T01:31:57.042ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 57fb3c2e-d67d-4572-97b9-71eed34b4a18 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22042025-03-25T01:31:57.042ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 57fb3c2e-d67d-4572-97b9-71eed34b4a18 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22052025-03-25T01:31:57.044ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2636 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 57fb3c2e-d67d-4572-97b9-71eed34b4a18 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22062025-03-25T01:31:58.064ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = abb08769-eede-495d-8a29-41bd6af074cb roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22072025-03-25T01:31:58.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = abb08769-eede-495d-8a29-41bd6af074cb resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22082025-03-25T01:31:58.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2889 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = abb08769-eede-495d-8a29-41bd6af074cb response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22092025-03-25T01:31:59.087ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2fa35a0a-840d-405a-bf51-fc96c53cb878 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22102025-03-25T01:31:59.087ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2fa35a0a-840d-405a-bf51-fc96c53cb878 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22112025-03-25T01:31:59.089ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2668 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 2fa35a0a-840d-405a-bf51-fc96c53cb878 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22122025-03-25T01:32:00.111ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 17bf4094-278d-4002-ae15-96c1ffe878f5 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22132025-03-25T01:32:00.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 17bf4094-278d-4002-ae15-96c1ffe878f5 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22142025-03-25T01:32:00.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2233 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 17bf4094-278d-4002-ae15-96c1ffe878f5 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22152025-03-25T01:32:01.132ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 32c14485-b992-4469-b9cd-c6be7426a96a roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22162025-03-25T01:32:01.133ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 32c14485-b992-4469-b9cd-c6be7426a96a resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22172025-03-25T01:32:01.134ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1791 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 32c14485-b992-4469-b9cd-c6be7426a96a response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
22182025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
22192025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
22202025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
22212025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22222025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
22232025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22242025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
22252025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
22262025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22272025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
22282025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
22292025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
22302025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
22312025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22322025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
22332025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22342025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22352025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
22362025-03-25T01:32:01.741ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22372025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22382025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
22392025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22402025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22412025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22422025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22432025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22442025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22452025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
22462025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
22472025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
22482025-03-25T01:32:01.742ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
22492025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22502025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22512025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
22522025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
22532025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
22542025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22552025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
22562025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
22572025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22582025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
22592025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
22602025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
22612025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22622025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
22632025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22642025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
22652025-03-25T01:32:01.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
22662025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22672025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22682025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22692025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
22702025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22712025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
22722025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22732025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
22742025-03-25T01:32:01.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22752025-03-25T01:32:01.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22762025-03-25T01:32:01.745ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1625, tv_nsec: 454194887 } })
22772025-03-25T01:32:01.745ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22782025-03-25T01:32:01.745ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.021328ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
22792025-03-25T01:32:01.745ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1077
22802025-03-25T01:32:01.746ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22812025-03-25T01:32:01.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22822025-03-25T01:32:01.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22832025-03-25T01:32:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22842025-03-25T01:32:01.748ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
22852025-03-25T01:32:01.748ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22862025-03-25T01:32:01.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22872025-03-25T01:32:01.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22882025-03-25T01:32:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22892025-03-25T01:32:01.749ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22902025-03-25T01:32:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22912025-03-25T01:32:01.750ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22922025-03-25T01:32:01.751ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1625, tv_nsec: 460096601 } })
22932025-03-25T01:32:01.751ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1077
22942025-03-25T01:32:01.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22952025-03-25T01:32:01.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22962025-03-25T01:32:01.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22972025-03-25T01:32:01.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
22982025-03-25T01:32:01.753ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22992025-03-25T01:32:01.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
23002025-03-25T01:32:01.754ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.94041ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
23012025-03-25T01:32:01.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23022025-03-25T01:32:01.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23032025-03-25T01:32:01.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23042025-03-25T01:32:01.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23052025-03-25T01:32:01.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23062025-03-25T01:32:01.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23072025-03-25T01:32:01.758ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23082025-03-25T01:32:01.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23092025-03-25T01:32:01.759ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 16.519557ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
23102025-03-25T01:32:01.760ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23112025-03-25T01:32:01.760ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23122025-03-25T01:32:01.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23132025-03-25T01:32:01.761ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-1768bc4c-c9ec-4fc7-a560-5ef338d49e99.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1625, tv_nsec: 469791900 } })
23142025-03-25T01:32:01.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23152025-03-25T01:32:01.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23162025-03-25T01:32:01.761ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1077 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
23172025-03-25T01:32:01.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23182025-03-25T01:32:01.762ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 19.828534ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23192025-03-25T01:32:01.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23202025-03-25T01:32:01.764ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23212025-03-25T01:32:01.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23222025-03-25T01:32:01.764ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23232025-03-25T01:32:01.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23242025-03-25T01:32:01.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23252025-03-25T01:32:01.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23262025-03-25T01:32:01.767ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23272025-03-25T01:32:01.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23282025-03-25T01:32:01.769ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
23292025-03-25T01:32:01.769ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 26.719892ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
23302025-03-25T01:32:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3ed5031c-b711-4951-ae01-aa99dea7c00b, lookup_type: ById(3ed5031c-b711-4951-ae01-aa99dea7c00b) } result = Ok(())
23312025-03-25T01:32:01.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23322025-03-25T01:32:01.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23332025-03-25T01:32:01.791ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b original_time_created = 2025-03-25 01:19:22.301325 UTC target_id = 3ed5031c-b711-4951-ae01-aa99dea7c00b time_created = 2025-03-25 01:19:22.301325 UTC
23342025-03-25T01:32:01.791ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 49.494618ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3ed5031c-b711-4951-ae01-aa99dea7c00b","time_created":"2025-03-25T01:19:22.301325Z"}
23352025-03-25T01:32:01.804ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23362025-03-25T01:32:01.805ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23372025-03-25T01:32:01.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23382025-03-25T01:32:01.808ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23392025-03-25T01:32:01.808ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
23402025-03-25T01:32:01.808ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23412025-03-25T01:32:01.809ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23422025-03-25T01:32:01.809ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23432025-03-25T01:32:01.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23442025-03-25T01:32:01.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23452025-03-25T01:32:01.810ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 67.301373ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
23462025-03-25T01:32:01.813ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23472025-03-25T01:32:01.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23482025-03-25T01:32:01.813ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23492025-03-25T01:32:01.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23502025-03-25T01:32:01.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23512025-03-25T01:32:01.818ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23522025-03-25T01:32:01.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23532025-03-25T01:32:01.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 1768bc4c-c9ec-4fc7-a560-5ef338d49e99, lookup_type: ById(1768bc4c-c9ec-4fc7-a560-5ef338d49e99) } result = Ok(())
23542025-03-25T01:32:01.818ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23552025-03-25T01:32:01.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
23562025-03-25T01:32:01.821ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23572025-03-25T01:32:01.821ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23582025-03-25T01:32:01.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
23592025-03-25T01:32:01.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
23602025-03-25T01:32:01.824ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23612025-03-25T01:32:01.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23622025-03-25T01:32:01.827ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 85.429223ms iteration = 26 status = 0
23632025-03-25T01:32:01.827ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23642025-03-25T01:32:01.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23652025-03-25T01:32:01.828ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:360
23662025-03-25T01:32:01.828ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23672025-03-25T01:32:01.828ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 86.829872ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
23682025-03-25T01:32:01.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23692025-03-25T01:32:01.830ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 87.903558ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23702025-03-25T01:32:01.831ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 88.217612ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
23712025-03-25T01:32:01.832ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
23722025-03-25T01:32:01.832ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23732025-03-25T01:32:01.833ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2374abc3-34de-4964-b8dc-5f1007b363a1", "content-length": "26", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
23742025-03-25T01:32:01.833ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23752025-03-25T01:32:01.836ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23762025-03-25T01:32:01.852ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1085 zone_address = fd00:1122:3344:101::2
23772025-03-25T01:32:01.852ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23782025-03-25T01:32:01.852ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "92058129-2072-419e-a125-da794a45c4d2", "content-length": "26", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
23792025-03-25T01:32:01.852ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1088 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23802025-03-25T01:32:01.852ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1085 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd zone_address = fd00:1122:3344:101::2
23812025-03-25T01:32:01.852ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23822025-03-25T01:32:01.852ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5488c3a0-1dca-4eb4-a7a4-55e4118556dd", "content-length": "26", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
23832025-03-25T01:32:01.852ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1088 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23842025-03-25T01:32:01.854ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23852025-03-25T01:32:01.854ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1122 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
23862025-03-25T01:32:01.858ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23872025-03-25T01:32:01.858ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
23882025-03-25T01:32:01.858ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23892025-03-25T01:32:01.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
23902025-03-25T01:32:01.859ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 118.271402ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23912025-03-25T01:32:01.911ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23922025-03-25T01:32:01.911ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
23932025-03-25T01:32:01.911ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 168.739224ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
23942025-03-25T01:32:01.913ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23952025-03-25T01:32:01.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23962025-03-25T01:32:01.915ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23972025-03-25T01:32:01.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23982025-03-25T01:32:01.917ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23992025-03-25T01:32:01.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24002025-03-25T01:32:01.919ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24012025-03-25T01:32:01.921ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24022025-03-25T01:32:01.921ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24032025-03-25T01:32:01.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
24042025-03-25T01:32:01.947ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 205.427519ms iteration = 26 status = {}
24052025-03-25T01:32:01.961ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
24062025-03-25T01:32:01.961ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 220.125309ms iteration = 29 status = {"physical_disks_added":0}
24072025-03-25T01:32:01.962ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d151dc08-01a3-466c-819a-f101f8804276", "content-length": "2", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24082025-03-25T01:32:01.963ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
24092025-03-25T01:32:01.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
24102025-03-25T01:32:01.975ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
24112025-03-25T01:32:01.975ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24122025-03-25T01:32:01.975ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f437ce1-202a-4ebc-bfa3-d936831d611f", "content-length": "372", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24132025-03-25T01:32:01.975ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24142025-03-25T01:32:01.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24152025-03-25T01:32:01.977ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 235.729933ms iteration = 26 status = null
24162025-03-25T01:32:01.990ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
24172025-03-25T01:32:01.990ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
24182025-03-25T01:32:01.990ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ef4b22df-d8d6-400e-bb19-599672543ae8", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24192025-03-25T01:32:01.990ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24202025-03-25T01:32:01.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24212025-03-25T01:32:01.992ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24222025-03-25T01:32:01.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24232025-03-25T01:32:01.995ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24242025-03-25T01:32:01.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24252025-03-25T01:32:02.037ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
24262025-03-25T01:32:02.037ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdc78395-80c7-4e53-a63c-36054d60974b", "content-length": "691", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24272025-03-25T01:32:02.037ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
24282025-03-25T01:32:02.037ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
24292025-03-25T01:32:02.037ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
24302025-03-25T01:32:02.039ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 298.068645ms iteration = 27 status = {}
24312025-03-25T01:32:02.045ZINFOnexus (BackgroundTasks): VPC route manager sled 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
24322025-03-25T01:32:02.045ZDEBGnexus (BackgroundTasks): client request SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24332025-03-25T01:32:02.045ZDEBGnexus (BackgroundTasks): client response SledAgent = 1768bc4c-c9ec-4fc7-a560-5ef338d49e99 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "daf80860-43a9-4810-b047-c7518d73e76c", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24342025-03-25T01:32:02.047ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 304.72832ms iteration = 27 status = {}
24352025-03-25T01:32:02.062ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24362025-03-25T01:32:02.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24372025-03-25T01:32:02.064ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 1a5160a0-1d63-486b-827e-4b7de2ad2079, rack_id: 16a011db-af85-49cc-8fa6-b468f46a78bd, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(80c078c2-e0e7-4e11-a9de-db9b8488eb2c) } rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
24382025-03-25T01:32:02.064ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24392025-03-25T01:32:02.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24402025-03-25T01:32:02.078ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
24412025-03-25T01:32:02.079ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "27491b15-2855-4389-81f3-4cfe1836b279", "content-length": "134", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24422025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1741 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
24432025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1768 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = 80c078c2-e0e7-4e11-a9de-db9b8488eb2c
24442025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
24452025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
24462025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24472025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24482025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24492025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1915 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24502025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24512025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1951 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24522025-03-25T01:32:02.079ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd
24532025-03-25T01:32:02.079ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24542025-03-25T01:32:02.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24552025-03-25T01:32:02.082ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
24562025-03-25T01:32:02.082ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "71a5b83e-0aeb-4953-9cdc-6f8d0757d440", "content-length": "2", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24572025-03-25T01:32:02.082ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
24582025-03-25T01:32:02.082ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2915815-44eb-47f5-bfaa-f9a0ef4938ba", "content-length": "51", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24592025-03-25T01:32:02.082ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch_location = Switch0
24602025-03-25T01:32:02.082ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24612025-03-25T01:32:02.208ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0550f93e-172a-4150-9a4d-3531e8daafb0", "date": "Tue, 25 Mar 2025 01:32:01 GMT"} })
24622025-03-25T01:32:02.208ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd switch = Switch0
24632025-03-25T01:32:02.208ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24642025-03-25T01:32:02.208ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f2ab2259-82d9-460a-af61-b7c5d0245649 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24652025-03-25T01:32:02.208ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f2ab2259-82d9-460a-af61-b7c5d0245649 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24662025-03-25T01:32:02.208ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 16a011db-af85-49cc-8fa6-b468f46a78bd result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdb3e0cb-d887-4990-a9f0-1f1f87bf9597", "content-length": "682", "date": "Tue, 25 Mar 2025 01:32:02 GMT"} })
24672025-03-25T01:32:02.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24682025-03-25T01:32:02.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24692025-03-25T01:32:02.210ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2384 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = f2ab2259-82d9-460a-af61-b7c5d0245649 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24702025-03-25T01:32:02.220ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24712025-03-25T01:32:02.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24722025-03-25T01:32:02.225ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24732025-03-25T01:32:02.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24742025-03-25T01:32:02.227ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24752025-03-25T01:32:02.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24762025-03-25T01:32:02.235ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 494.187281ms iteration = 26 status = {}
24772025-03-25T01:32:03.238ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c66ac06a-8840-4502-bc41-03aea2bdfd39 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24782025-03-25T01:32:03.238ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c66ac06a-8840-4502-bc41-03aea2bdfd39 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24792025-03-25T01:32:03.240ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2231 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = c66ac06a-8840-4502-bc41-03aea2bdfd39 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24802025-03-25T01:32:04.262ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 643187a5-9ab5-4ea4-9c2e-17e1d10a630f roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24812025-03-25T01:32:04.262ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 643187a5-9ab5-4ea4-9c2e-17e1d10a630f resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24822025-03-25T01:32:04.264ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2323 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 643187a5-9ab5-4ea4-9c2e-17e1d10a630f response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24832025-03-25T01:32:05.285ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cd99b597-1673-4c6d-b2f1-8247f80a4fc3 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24842025-03-25T01:32:05.285ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cd99b597-1673-4c6d-b2f1-8247f80a4fc3 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24852025-03-25T01:32:05.287ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 1942 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = cd99b597-1673-4c6d-b2f1-8247f80a4fc3 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24862025-03-25T01:32:05.769ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 524 local_addr = [fd00:1122:3344:101::b]:37373 method = GET remote_addr = [fd00:1122:3344:101::d]:48741 req_id = 82bfd22a-ab39-4b74-9cba-b6c35f0cb3a9 response_code = 200 uri = /3382b509-413c-4e85-971f-4b24ca64ae0c
24872025-03-25T01:32:06.306ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 783d2128-5ff1-463c-a8a6-1466700c3d33 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24882025-03-25T01:32:06.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 783d2128-5ff1-463c-a8a6-1466700c3d33 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24892025-03-25T01:32:06.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2873 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 783d2128-5ff1-463c-a8a6-1466700c3d33 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24902025-03-25T01:32:07.329ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = a78033e3-e2ea-415c-89b6-b2db631b6bb9 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24912025-03-25T01:32:07.329ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = a78033e3-e2ea-415c-89b6-b2db631b6bb9 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24922025-03-25T01:32:07.331ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2349 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = a78033e3-e2ea-415c-89b6-b2db631b6bb9 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24932025-03-25T01:32:08.352ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 20a1590a-3a59-4802-943a-1d963b4f39e0 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24942025-03-25T01:32:08.352ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 20a1590a-3a59-4802-943a-1d963b4f39e0 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24952025-03-25T01:32:08.354ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2305 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 20a1590a-3a59-4802-943a-1d963b4f39e0 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24962025-03-25T01:32:09.372ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 236d60e6-9ec0-4a52-a9d1-a485af5776d6 roles = RoleSet { roles: {} } uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24972025-03-25T01:32:09.372ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 236d60e6-9ec0-4a52-a9d1-a485af5776d6 resource = Database result = Ok(()) uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99
24982025-03-25T01:32:09.374ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 2385 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:36658 req_id = 236d60e6-9ec0-4a52-a9d1-a485af5776d6 response_code = 200 uri = /probes/1768bc4c-c9ec-4fc7-a560-5ef338d49e99