Max level shown:
12024-11-19T01:45:02.921ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7bb95de0-a07c-4df3-ac9a-e490e5f36e31 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
22024-11-19T01:45:02.922ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7bb95de0-a07c-4df3-ac9a-e490e5f36e31 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32024-11-19T01:45:02.923ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2106 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7bb95de0-a07c-4df3-ac9a-e490e5f36e31 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
42024-11-19T01:45:02.979ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3ac0ddea-468c-478c-a857-3436d3570b4c uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092 vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-19T01:45:02.979423216Z }
52024-11-19T01:45:02.979ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3ac0ddea-468c-478c-a857-3436d3570b4c roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
62024-11-19T01:45:02.980ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3ac0ddea-468c-478c-a857-3436d3570b4c resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
72024-11-19T01:45:02.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 14370 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3ac0ddea-468c-478c-a857-3436d3570b4c response_code = 204 uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
82024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 26 reason = Timeout
92024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
102024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
112024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
122024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
132024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
142024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
152024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
162024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
172024-11-19T01:45:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
182024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
192024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
202024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
212024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
222024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
232024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
242024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
252024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
262024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
272024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
282024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
292024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
302024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
312024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
322024-11-19T01:45:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
332024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
342024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
352024-11-19T01:45:03.531ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
362024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
372024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
382024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
392024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
402024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
412024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
422024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
432024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
442024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
452024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
462024-11-19T01:45:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
472024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
482024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
492024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
502024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
512024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
522024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
532024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
542024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
552024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
562024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
572024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
582024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
592024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.168347ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
602024-11-19T01:45:03.532ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 700148909 } })
612024-11-19T01:45:03.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
622024-11-19T01:45:03.532ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 700401503 } })
632024-11-19T01:45:03.532ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
642024-11-19T01:45:03.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
652024-11-19T01:45:03.533ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
662024-11-19T01:45:03.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
672024-11-19T01:45:03.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
682024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
692024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
702024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
712024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
722024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
732024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
742024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
752024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
762024-11-19T01:45:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
772024-11-19T01:45:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
782024-11-19T01:45:03.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
792024-11-19T01:45:03.537ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
802024-11-19T01:45:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
812024-11-19T01:45:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
822024-11-19T01:45:03.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
832024-11-19T01:45:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
842024-11-19T01:45:03.538ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
852024-11-19T01:45:03.539ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
862024-11-19T01:45:03.539ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
872024-11-19T01:45:03.539ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
882024-11-19T01:45:03.539ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
892024-11-19T01:45:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
902024-11-19T01:45:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
912024-11-19T01:45:03.540ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
922024-11-19T01:45:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
932024-11-19T01:45:03.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
942024-11-19T01:45:03.541ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
952024-11-19T01:45:03.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
962024-11-19T01:45:03.542ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
972024-11-19T01:45:03.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
982024-11-19T01:45:03.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
992024-11-19T01:45:03.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1002024-11-19T01:45:03.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1012024-11-19T01:45:03.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1022024-11-19T01:45:03.545ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1753, tv_nsec: 712610842 } })
1032024-11-19T01:45:03.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1042024-11-19T01:45:03.545ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
1052024-11-19T01:45:03.545ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 13.734126ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
1062024-11-19T01:45:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1072024-11-19T01:45:03.545ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 15.006259ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
1082024-11-19T01:45:03.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1092024-11-19T01:45:03.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1102024-11-19T01:45:03.547ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1112024-11-19T01:45:03.547ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
1122024-11-19T01:45:03.547ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1132024-11-19T01:45:03.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
1142024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
1152024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1162024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1172024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1182024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1192024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1202024-11-19T01:45:03.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1212024-11-19T01:45:03.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1222024-11-19T01:45:03.553ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1232024-11-19T01:45:03.553ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
1242024-11-19T01:45:03.553ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 21.822021ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
1252024-11-19T01:45:03.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 36.879609ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
1262024-11-19T01:45:03.569ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1272024-11-19T01:45:03.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1282024-11-19T01:45:03.571ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1292024-11-19T01:45:03.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1302024-11-19T01:45:03.573ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1312024-11-19T01:45:03.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1322024-11-19T01:45:03.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 42.703083ms iteration = 26 status = {"errors":[],"records_set_to_done":[]}
1332024-11-19T01:45:03.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1342024-11-19T01:45:03.580ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 48.348274ms iteration = 26 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
1352024-11-19T01:45:03.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
1362024-11-19T01:45:03.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1372024-11-19T01:45:03.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
1382024-11-19T01:45:03.644ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 113.593782ms iteration = 30 status = {"physical_disks_added":0}
1392024-11-19T01:45:03.650ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1402024-11-19T01:45:03.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1412024-11-19T01:45:03.657ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
1422024-11-19T01:45:03.657ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1432024-11-19T01:45:03.658ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd59a684-08a4-46a5-a4d0-52d0277752eb", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:03 GMT"} })
1442024-11-19T01:45:03.658ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1452024-11-19T01:45:03.660ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1462024-11-19T01:45:03.669ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
1472024-11-19T01:45:03.669ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1482024-11-19T01:45:03.669ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eddf6373-c024-4355-b239-c5e3182cc067", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:03 GMT"} })
1492024-11-19T01:45:03.669ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1502024-11-19T01:45:03.671ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
1512024-11-19T01:45:03.687ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/f29067fd-e978-4a5d-8f3c-f6f062710092/state
1522024-11-19T01:45:03.691ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532024-11-19T01:45:03.691ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1542024-11-19T01:45:03.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1552024-11-19T01:45:03.692ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1562024-11-19T01:45:03.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1572024-11-19T01:45:03.693ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1582024-11-19T01:45:03.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1592024-11-19T01:45:03.694ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1602024-11-19T01:45:03.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
1612024-11-19T01:45:03.696ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1622024-11-19T01:45:03.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
1632024-11-19T01:45:03.696ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1642024-11-19T01:45:03.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
1652024-11-19T01:45:03.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1662024-11-19T01:45:03.701ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
1672024-11-19T01:45:03.701ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 172.442124ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
1682024-11-19T01:45:03.726ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 194.6106ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
1692024-11-19T01:45:03.789ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 260.181842ms iteration = 26 status = {}
1702024-11-19T01:45:03.796ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
1712024-11-19T01:45:03.797ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a99485a-990f-4f73-ac7f-9ac7a7e6a68f", "content-length": "1", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
1722024-11-19T01:45:03.798ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
1732024-11-19T01:45:03.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
1742024-11-19T01:45:03.799ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
1752024-11-19T01:45:03.799ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
1762024-11-19T01:45:03.799ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
1772024-11-19T01:45:03.799ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1782024-11-19T01:45:03.799ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a7b790a-4cbf-4b16-8f7f-4c692b9fe2e2", "content-length": "585", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
1792024-11-19T01:45:03.799ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1802024-11-19T01:45:03.799ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd0f7117-27e0-4b6f-9611-cc6f5a59a8f7", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:03 GMT"} })
1812024-11-19T01:45:03.799ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1822024-11-19T01:45:03.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1832024-11-19T01:45:03.801ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
1842024-11-19T01:45:03.801ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 272.318229ms iteration = 26 status = null
1852024-11-19T01:45:03.801ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 271.224791ms iteration = 26 status = 0
1862024-11-19T01:45:03.840ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {1bf5f9af-1594-4e08-8b77-efe08afaca00: {10.151.1.175: {425ca83c-cc06-4aed-b1b3-e1eae15d7447}, 10.151.1.176: {425ca83c-cc06-4aed-b1b3-e1eae15d7447}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
1872024-11-19T01:45:03.840ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
1882024-11-19T01:45:03.863ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
1892024-11-19T01:45:03.910ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "42f345b2-a330-48f0-97cf-7a070e425687", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
1902024-11-19T01:45:03.910ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1912024-11-19T01:45:03.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1922024-11-19T01:45:03.911ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "22d0eaad-c590-4b03-ac04-0bb98ad123a4", "content-length": "811", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
1932024-11-19T01:45:03.911ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(10299830) }} sled = buskin
1942024-11-19T01:45:03.911ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
1952024-11-19T01:45:03.911ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
1962024-11-19T01:45:03.912ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 382.060582ms iteration = 26 status = {}
1972024-11-19T01:45:03.913ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1982024-11-19T01:45:03.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1992024-11-19T01:45:03.916ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2002024-11-19T01:45:03.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2012024-11-19T01:45:03.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2022024-11-19T01:45:03.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2032024-11-19T01:45:03.922ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2042024-11-19T01:45:03.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2052024-11-19T01:45:03.924ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2062024-11-19T01:45:03.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2072024-11-19T01:45:03.925ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2082024-11-19T01:45:03.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2092024-11-19T01:45:03.925ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
2102024-11-19T01:45:03.925ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2112024-11-19T01:45:03.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2122024-11-19T01:45:03.928ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2132024-11-19T01:45:03.928ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
2142024-11-19T01:45:03.928ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
2152024-11-19T01:45:03.928ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7e063a8c-e525-40e4-ac51-46ddd4e1da4c", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
2162024-11-19T01:45:03.930ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 398.976195ms iteration = 27 status = {}
2172024-11-19T01:45:03.935ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
2182024-11-19T01:45:03.935ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "57b858c5-19fd-464a-830c-2d17046e96b6", "content-length": "134", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
2192024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
2202024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
2212024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
2222024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
2232024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2242024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2252024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2262024-11-19T01:45:03.935ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2272024-11-19T01:45:03.936ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
2282024-11-19T01:45:03.936ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
2292024-11-19T01:45:03.936ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
2302024-11-19T01:45:03.936ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2312024-11-19T01:45:03.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2322024-11-19T01:45:03.938ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
2332024-11-19T01:45:03.938ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f97a804-9e97-41f7-b641-45e93cc726f9", "content-length": "2", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
2342024-11-19T01:45:03.938ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
2352024-11-19T01:45:03.938ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "0272614c-ea48-4727-8869-a9b89780dec1", "content-length": "51", "date": "Tue, 19 Nov 2024 01:45:02 GMT"} })
2362024-11-19T01:45:03.938ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
2372024-11-19T01:45:03.938ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
2382024-11-19T01:45:04.036ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "97a6d227-9b99-4fb5-98c8-eec9fdbea042", "date": "Tue, 19 Nov 2024 01:45:04 GMT"} })
2392024-11-19T01:45:04.036ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
2402024-11-19T01:45:04.036ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
2412024-11-19T01:45:04.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 86ef913f-6432-4eec-bc1a-eacd26be329f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
2422024-11-19T01:45:04.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 86ef913f-6432-4eec-bc1a-eacd26be329f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
2432024-11-19T01:45:04.036ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = dc5ada9d-10de-4f7f-bdd5-d7c90b4bcc7d roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/9?limit=100
2442024-11-19T01:45:04.036ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "91cca411-b661-4270-9379-43a19b35c0e5", "content-length": "682", "date": "Tue, 19 Nov 2024 01:45:04 GMT"} })
2452024-11-19T01:45:04.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2462024-11-19T01:45:04.036ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = dc5ada9d-10de-4f7f-bdd5-d7c90b4bcc7d resource = Database result = Ok(()) uri = /nat/ipv4/changeset/9?limit=100
2472024-11-19T01:45:04.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2482024-11-19T01:45:04.038ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2475 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 86ef913f-6432-4eec-bc1a-eacd26be329f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
2492024-11-19T01:45:04.040ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3391 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = dc5ada9d-10de-4f7f-bdd5-d7c90b4bcc7d response_code = 200 uri = /nat/ipv4/changeset/9?limit=100
2502024-11-19T01:45:04.040ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2512024-11-19T01:45:04.040ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = c96a3d6b-a3d8-4bbb-9a37-7c1d95ed1912 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
2522024-11-19T01:45:04.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = c96a3d6b-a3d8-4bbb-9a37-7c1d95ed1912 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
2532024-11-19T01:45:04.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2542024-11-19T01:45:04.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2954 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = c96a3d6b-a3d8-4bbb-9a37-7c1d95ed1912 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
2552024-11-19T01:45:04.044ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2562024-11-19T01:45:04.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2572024-11-19T01:45:04.046ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2582024-11-19T01:45:04.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2592024-11-19T01:45:04.053ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 522.87396ms iteration = 26 status = {}
2602024-11-19T01:45:05.045ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 0cf92937-13b1-445e-be2d-38c0dc92432c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
2612024-11-19T01:45:05.045ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 0cf92937-13b1-445e-be2d-38c0dc92432c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
2622024-11-19T01:45:05.048ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2886 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 0cf92937-13b1-445e-be2d-38c0dc92432c response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
2632024-11-19T01:45:05.058ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37fd63bf-8936-488b-9e1f-695424aae3a6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
2642024-11-19T01:45:05.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37fd63bf-8936-488b-9e1f-695424aae3a6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
2652024-11-19T01:45:05.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37fd63bf-8936-488b-9e1f-695424aae3a6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
2662024-11-19T01:45:05.489ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-19T01:45:05.488982383Z }
2672024-11-19T01:45:05.489ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2682024-11-19T01:45:05.489ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2692024-11-19T01:45:05.526ZDEBGnexus (dropshot_internal): new VMM runtime state from sled agent requires an instance-update saga actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_in_needs_update = false migration_out_needs_update = false propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092 vmm_needs_update = true
2702024-11-19T01:45:05.526ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2712024-11-19T01:45:05.526ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2722024-11-19T01:45:05.528ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2732024-11-19T01:45:05.528ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2742024-11-19T01:45:05.530ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2752024-11-19T01:45:05.531ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2762024-11-19T01:45:05.533ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2772024-11-19T01:45:05.533ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2782024-11-19T01:45:05.534ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2792024-11-19T01:45:05.534ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2802024-11-19T01:45:05.535ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2812024-11-19T01:45:05.536ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2822024-11-19T01:45:05.537ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2832024-11-19T01:45:05.540ZDEBGnexus (dropshot_internal): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, lookup_type: ById(7ae1255b-c149-4b2a-8f4e-7489cdeaaefa) }, key: 43ed357a-1904-4080-ab4e-07192398e688, lookup_type: ById(43ed357a-1904-4080-ab4e-07192398e688) }, key: 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a, lookup_type: ById(9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a) } result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2842024-11-19T01:45:05.540ZINFOnexus (dropshot_internal): starting update saga for 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:1618 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-19T01:45:05.488982383Z }
2852024-11-19T01:45:05.540ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51070 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de response_code = 204 uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2862024-11-19T01:45:05.540ZDEBGnexus (dropshot_internal): preparing instance-update saga for 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a... actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7201ab7b-7a95-4410-b12f-873e4b8ba2de uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2872024-11-19T01:45:05.540ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update
2882024-11-19T01:45:05.540ZINFOnexus (dropshot_internal): received new VMM runtime state from sled agent actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true file = nexus/src/app/instance.rs:2151 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfe1cf30-750d-4ccc-bcf2-17564ef92108 uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-19T01:45:05.488982383Z }
2892024-11-19T01:45:05.540ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfe1cf30-750d-4ccc-bcf2-17564ef92108 roles = RoleSet { roles: {} } uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2902024-11-19T01:45:05.540ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfe1cf30-750d-4ccc-bcf2-17564ef92108 resource = Database result = Ok(()) uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2912024-11-19T01:45:05.542ZINFOnexus (SEC): saga create dag = {"end_node":4,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[3,0,null],[2,4,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"start_instance_update.lock_instance","label":"LockInstance","name":"updater_lock"}},{"Action":{"action_name":"start_instance_update.fetch_state_and_start_real_saga","label":"FetchStateAndStartRealSaga","name":"state"}},{"Start":{"params":{"authz_instance":{"key":"9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a","lookup_type":{"ById":"9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a"},"parent":{"key":"43ed357a-1904-4080-ab4e-07192398e688","lookup_type":{"ById":"43ed357a-1904-4080-ab4e-07192398e688"},"parent":{"key":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","lookup_type":{"ById":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa"},"parent":null}}},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}}}}},"End"]},"saga_name":"start-instance-update","start_node":3} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
2922024-11-19T01:45:05.542ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update
2932024-11-19T01:45:05.577ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37101 local_addr = [fd00:1122:3344:101::b]:12221 method = PUT remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfe1cf30-750d-4ccc-bcf2-17564ef92108 response_code = 204 uri = /vmms/f29067fd-e978-4a5d-8f3c-f6f062710092
2942024-11-19T01:45:05.578ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/f29067fd-e978-4a5d-8f3c-f6f062710092/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "6539d526-6941-4151-8a8d-737ad84963c7", "content-length": "132", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
2952024-11-19T01:45:05.579ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update
2962024-11-19T01:45:05.579ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
2972024-11-19T01:45:05.579ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
2982024-11-19T01:45:05.579ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
2992024-11-19T01:45:05.580ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a sled_id = 302245ee-8785-4baf-86ed-0823410907ec state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-19T01:45:05.488982383Z }, migration_in: None, migration_out: None } vmm_id = f29067fd-e978-4a5d-8f3c-f6f062710092
3002024-11-19T01:45:05.580ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2151 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 sled_id = 302245ee-8785-4baf-86ed-0823410907ec vmm_id = f29067fd-e978-4a5d-8f3c-f6f062710092 vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-19T01:45:05.488982383Z }
3012024-11-19T01:45:05.580ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a roles = RoleSet { roles: {} } sled_id = 302245ee-8785-4baf-86ed-0823410907ec vmm_id = f29067fd-e978-4a5d-8f3c-f6f062710092
3022024-11-19T01:45:05.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a resource = Database result = Ok(()) sled_id = 302245ee-8785-4baf-86ed-0823410907ec vmm_id = f29067fd-e978-4a5d-8f3c-f6f062710092
3032024-11-19T01:45:05.584ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3042024-11-19T01:45:05.584ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
3052024-11-19T01:45:05.587ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
3062024-11-19T01:45:05.587ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3072024-11-19T01:45:05.587ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
3082024-11-19T01:45:05.589ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 2.057588765s iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"destroyed":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
3092024-11-19T01:45:05.591ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3102024-11-19T01:45:05.591ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("283e2008-df56-4825-ae30-ae75cee4007f")) node_id = 0 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
3112024-11-19T01:45:05.595ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3122024-11-19T01:45:05.595ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
3132024-11-19T01:45:05.601ZINFOnexus (SagaExecutor): instance update: attempting to lock instance file = nexus/src/app/sagas/instance_update/start.rs:93 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update
3142024-11-19T01:45:05.601ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3152024-11-19T01:45:05.601ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3162024-11-19T01:45:05.603ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3172024-11-19T01:45:05.603ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3182024-11-19T01:45:05.604ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3192024-11-19T01:45:05.605ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3202024-11-19T01:45:05.606ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3212024-11-19T01:45:05.606ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3222024-11-19T01:45:05.608ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3232024-11-19T01:45:05.608ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3242024-11-19T01:45:05.609ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3252024-11-19T01:45:05.609ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3262024-11-19T01:45:05.610ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3272024-11-19T01:45:05.613ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, lookup_type: ById(7ae1255b-c149-4b2a-8f4e-7489cdeaaefa) }, key: 43ed357a-1904-4080-ab4e-07192398e688, lookup_type: ById(43ed357a-1904-4080-ab4e-07192398e688) }, key: 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a, lookup_type: ById(9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a) } result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3282024-11-19T01:45:05.613ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance updater_id = 283e2008-df56-4825-ae30-ae75cee4007f
3292024-11-19T01:45:05.613ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3302024-11-19T01:45:05.614ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3312024-11-19T01:45:05.626ZDEBGnexus (SagaExecutor): attempting to acquire instance updater lock actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true current_gen = Generation(Generation(1)) instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance updater_id = 283e2008-df56-4825-ae30-ae75cee4007f
3322024-11-19T01:45:05.626ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3332024-11-19T01:45:05.626ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance
3342024-11-19T01:45:05.629ZDEBGnexus (SagaExecutor): instance updater lock acquired! actor_id = 001de000-05e4-4000-8000-000000000002 already_locked = true authenticated = true instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a locked_gen = Generation(Generation(2)) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = LockInstance updater_id = 283e2008-df56-4825-ae30-ae75cee4007f
3352024-11-19T01:45:05.632ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3362024-11-19T01:45:05.632ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(2), "updater_id": String("283e2008-df56-4825-ae30-ae75cee4007f")}) node_id = 1 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
3372024-11-19T01:45:05.637ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3382024-11-19T01:45:05.637ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
3392024-11-19T01:45:05.643ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3402024-11-19T01:45:05.643ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3412024-11-19T01:45:05.645ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3422024-11-19T01:45:05.645ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3432024-11-19T01:45:05.646ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3442024-11-19T01:45:05.646ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3452024-11-19T01:45:05.647ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3462024-11-19T01:45:05.650ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, lookup_type: ById(7ae1255b-c149-4b2a-8f4e-7489cdeaaefa) }, key: 43ed357a-1904-4080-ab4e-07192398e688, lookup_type: ById(43ed357a-1904-4080-ab4e-07192398e688) }, key: 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a, lookup_type: ById(9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a) } result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3472024-11-19T01:45:05.650ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3482024-11-19T01:45:05.650ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_node = FetchStateAndStartRealSaga
3492024-11-19T01:45:05.659ZINFOnexus (SagaExecutor): instance update: starting real update saga... current.active_vmm = Some(Vmm { id: f29067fd-e978-4a5d-8f3c-f6f062710092, time_created: 2024-11-19T01:44:44.790748Z, time_deleted: None, instance_id: 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a, sled_id: 302245ee-8785-4baf-86ed-0823410907ec, propolis_ip: V6(Ipv6Network { addr: fd00:1122:3344:101::1:0, prefix: 128 }), propolis_port: SqlU16(12400), runtime: VmmRuntimeState { time_state_updated: 2024-11-19T01:45:05.488982Z, gen: Generation(Generation(8)), state: Destroyed } }) current.migration = None current.runtime_state = InstanceRuntimeState { time_updated: 2024-11-19T01:44:42.710477Z, gen: Generation(Generation(3)), propolis_id: Some(f29067fd-e978-4a5d-8f3c-f6f062710092), dst_propolis_id: None, migration_id: None, nexus_state: Vmm, time_last_auto_restarted: None } current.target_vmm = None file = nexus/src/app/sagas/instance_update/start.rs:180 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update update.deprovision = true update.destroy_active_vmm = Some(f29067fd-e978-4a5d-8f3c-f6f062710092 (propolis)) update.destroy_target_vmm = None update.network_config_update = Some(Delete) update.new_runtime_state = InstanceRuntimeState { time_updated: 2024-11-19T01:45:05.659257278Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None }
3502024-11-19T01:45:05.661ZINFOnexus (SagaExecutor): preparing saga file = nexus/src/app/saga.rs:255 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
3512024-11-19T01:45:05.661ZINFOnexus (SEC): saga create dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[3,4,null],[4,5,null],[5,6,null],[6,7,null],[7,8,null],[9,10,null],[8,9,null],[10,11,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"common.uuid_generate","label":"GenerateInstanceLockId","name":"saga_instance_lock_id"}},{"Action":{"action_name":"instance_update.become_updater","label":"BecomeUpdater","name":"updater_lock"}},{"Constant":{"name":"network_config_update","value":"Delete"}},{"Action":{"action_name":"instance_update.update_network_config","label":"UpdateNetworkConfig","name":"update_network_config"}},{"Action":{"action_name":"instance_update.release_virtual_provisioning","label":"ReleaseVirtualProvisioning","name":"release_virtual_provisioning"}},{"Action":{"action_name":"instance_update.unassign_oximeter_producer","label":"UnassignOximeterProducer","name":"unassign_oximeter_producer"}},{"Action":{"action_name":"instance_update.commit_instance_updates","label":"CommitInstanceUpdates","name":"commit_instance_updates"}},{"Constant":{"name":"destroy_active_vmm_params","value":{"instance_id":"9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a","serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"vmm_id":"f29067fd-e978-4a5d-8f3c-f6f062710092"}}},{"SubsagaStart":{"params_node_name":"destroy_active_vmm_params","saga_name":"destroy_active_vmm"}},{"Action":{"action_name":"destroy_vmm.release_sled_resources","label":"ReleaseSledResources","name":"release_sled_resources"}},{"Action":{"action_name":"destroy_vmm.mark_vmm_deleted","label":"MarkVmmDeleted","name":"mark_vmm_deleted"}},{"SubsagaEnd":{"name":"active_vmm_destroyed"}},{"Action":{"action_name":"instance_update.chain_successor_saga","label":"ChainSuccessorSaga","name":"chain_successor_saga"}},{"Start":{"params":{"authz_instance":{"key":"9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a","lookup_type":{"ById":"9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a"},"parent":{"key":"43ed357a-1904-4080-ab4e-07192398e688","lookup_type":{"ById":"43ed357a-1904-4080-ab4e-07192398e688"},"parent":{"key":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","lookup_type":{"ById":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa"},"parent":null}}},"orig_lock":{"locked_gen":2,"updater_id":"283e2008-df56-4825-ae30-ae75cee4007f"},"serialized_authn":{"kind":{"Authenticated":[{"actor":{"UserBuiltin":{"user_builtin_id":"001de000-05e4-4000-8000-000000000002"}}},null]}},"update":{"deprovision":{"cpus_diff":2,"project_id":"43ed357a-1904-4080-ab4e-07192398e688","ram_diff":1073741824},"destroy_active_vmm":"f29067fd-e978-4a5d-8f3c-f6f062710092","destroy_target_vmm":null,"network_config":"Delete","new_runtime":{"dst_propolis_id":null,"gen":4,"migration_id":null,"nexus_state":"NoVmm","propolis_id":null,"time_last_auto_restarted":null,"time_updated":"2024-11-19T01:45:05.659257278Z"}}}}},"End"]},"saga_name":"instance-update","start_node":13} file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1146 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3522024-11-19T01:45:05.661ZINFOnexus (SecStore): creating saga file = nexus/db-queries/src/db/sec_store.rs:50 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
3532024-11-19T01:45:05.692ZINFOnexus (SagaExecutor): starting saga file = nexus/src/app/saga.rs:353 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
3542024-11-19T01:45:05.692ZINFOnexus (SEC): saga start file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:967 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3552024-11-19T01:45:05.692ZDEBGnexus (SEC): saga log event new_state = N013 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3562024-11-19T01:45:05.692ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 13 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3572024-11-19T01:45:05.696ZDEBGnexus (SEC): saga log event new_state = N013 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3582024-11-19T01:45:05.696ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 13 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3592024-11-19T01:45:05.699ZDEBGnexus (SEC): saga log event new_state = N000 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3602024-11-19T01:45:05.699ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 0 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3612024-11-19T01:45:05.702ZDEBGnexus (SEC): saga log event new_state = N000 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3622024-11-19T01:45:05.702ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("9b1e1baf-58de-4735-9f10-8e295a116d78")) node_id = 0 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3632024-11-19T01:45:05.705ZDEBGnexus (SEC): saga log event new_state = N001 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3642024-11-19T01:45:05.705ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 1 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3652024-11-19T01:45:05.710ZDEBGnexus (SagaExecutor): instance update: trying to become instance updater... instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a parent_lock = UpdaterLock { updater_id: 283e2008-df56-4825-ae30-ae75cee4007f, locked_gen: Generation(Generation(2)) } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
3662024-11-19T01:45:05.710ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = BecomeUpdater
3672024-11-19T01:45:05.711ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = BecomeUpdater
3682024-11-19T01:45:05.719ZDEBGnexus (SagaExecutor): inherited lock from 283e2008-df56-4825-ae30-ae75cee4007f to 9b1e1baf-58de-4735-9f10-8e295a116d78 actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a locked_gen = Generation(Generation(3)) parent_gen = Generation(Generation(2)) parent_id = 283e2008-df56-4825-ae30-ae75cee4007f saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = BecomeUpdater updater_id = 9b1e1baf-58de-4735-9f10-8e295a116d78
3692024-11-19T01:45:05.719ZINFOnexus (SagaExecutor): instance_update: Now, I am become Updater, the destroyer of VMMs. file = nexus/src/app/sagas/instance_update/mod.rs:964 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
3702024-11-19T01:45:05.719ZDEBGnexus (SEC): saga log event new_state = N001 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3712024-11-19T01:45:05.719ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"locked_gen": Number(3), "updater_id": String("9b1e1baf-58de-4735-9f10-8e295a116d78")}) node_id = 1 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3722024-11-19T01:45:05.722ZDEBGnexus (SEC): saga log event new_state = N002 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3732024-11-19T01:45:05.722ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 2 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3742024-11-19T01:45:05.725ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3752024-11-19T01:45:05.725ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(String("Delete")) node_id = 2 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3762024-11-19T01:45:05.728ZDEBGnexus (SEC): saga log event new_state = N003 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
3772024-11-19T01:45:05.728ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 3 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
3782024-11-19T01:45:05.733ZINFOnexus (SagaExecutor): instance update: deleting network config file = nexus/src/app/sagas/instance_update/mod.rs:1017 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
3792024-11-19T01:45:05.733ZINFOnexus: deleting instance dpd configuration file = nexus/src/app/instance_network.rs:548 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a
3802024-11-19T01:45:05.735ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UpdateNetworkConfig
3812024-11-19T01:45:05.735ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UpdateNetworkConfig
3822024-11-19T01:45:05.738ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UpdateNetworkConfig
3832024-11-19T01:45:05.738ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UpdateNetworkConfig
3842024-11-19T01:45:05.740ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
3852024-11-19T01:45:05.740ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UpdateNetworkConfig
3862024-11-19T01:45:05.740ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UpdateNetworkConfig
3872024-11-19T01:45:05.742ZWARNnexus: no matching nat entries to soft delete file = nexus/src/app/instance_network.rs:711
3882024-11-19T01:45:05.742ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3892024-11-19T01:45:05.742ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3902024-11-19T01:45:05.743ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
3912024-11-19T01:45:05.744ZDEBGnexus (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Fleet result = Ok(())
3922024-11-19T01:45:05.745ZDEBGnexus (InstanceAllocator): roles roles = RoleSet { roles: {} }
3932024-11-19T01:45:05.745ZDEBGnexus (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
3942024-11-19T01:45:05.749ZDEBGnexus: notifying dendrite of updates instance_id = Some(9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a (instance)) switch = switch0
3952024-11-19T01:45:05.750ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1755, tv_nsec: 917871436 } })
3962024-11-19T01:45:05.750ZINFOnexus: Determining switch slots managed by switch zones file = nexus/src/app/mod.rs:1131
3972024-11-19T01:45:05.780ZINFOnexus: determining switch slot managed by dendrite zone file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
3982024-11-19T01:45:05.780ZDEBGnexus (MgsClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3992024-11-19T01:45:05.781ZDEBGnexus (MgsClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a2a1cf8-51e2-47b0-9e91-6b076f574c1e", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
4002024-11-19T01:45:05.781ZINFOnexus: identified switch slot for dendrite zone file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4012024-11-19T01:45:05.782ZINFOnexus: completed mapping dendrite zones to switch slots file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4022024-11-19T01:45:05.811ZDEBGnexus (DpdClient): client request body = None method = POST uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger
4032024-11-19T01:45:05.811ZDEBGnexus (DpdClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/trigger", status: 200, headers: {"content-type": "application/json", "x-request-id": "0347f60c-18f5-45c7-b15e-b14a3b5eea51", "content-length": "4", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
4042024-11-19T01:45:05.813ZDEBGnexus (SEC): saga log event new_state = N003 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4052024-11-19T01:45:05.813ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 3 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4062024-11-19T01:45:05.817ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4072024-11-19T01:45:05.817ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4082024-11-19T01:45:05.820ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
4092024-11-19T01:45:05.820ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ReleaseVirtualProvisioning
4102024-11-19T01:45:05.831ZINFOnexus (SagaExecutor): instance update (no VMM): deallocated virtual provisioning resources file = nexus/src/app/sagas/instance_update/mod.rs:1092 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a records_deleted = [VirtualProvisioningCollection { id: 001de000-1334-4000-8000-000000000000, time_modified: Some(2024-11-19T01:45:05.823018Z), collection_type: "Fleet", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 43ed357a-1904-4080-ab4e-07192398e688, time_modified: Some(2024-11-19T01:45:05.823018Z), collection_type: "Project", virtual_disk_bytes_provisioned: ByteCount(ByteCount(2147483648)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }, VirtualProvisioningCollection { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, time_modified: Some(2024-11-19T01:45:05.823018Z), collection_type: "Silo", virtual_disk_bytes_provisioned: ByteCount(ByteCount(6442450944)), cpus_provisioned: 0, ram_provisioned: ByteCount(ByteCount(0)) }] saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4112024-11-19T01:45:05.831ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4122024-11-19T01:45:05.831ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4132024-11-19T01:45:05.835ZDEBGnexus (SEC): saga log event new_state = N005 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4142024-11-19T01:45:05.835ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 5 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4152024-11-19T01:45:05.839ZINFOnexus (SagaExecutor): instance update (no VMM): unassigning oximeter producer file = nexus/src/app/sagas/instance_update/mod.rs:1127 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4162024-11-19T01:45:05.841ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UnassignOximeterProducer
4172024-11-19T01:45:05.841ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UnassignOximeterProducer
4182024-11-19T01:45:05.846ZDEBGnexus (SagaExecutor): deleted metric producer assignment collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 producer_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4192024-11-19T01:45:05.846ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UnassignOximeterProducer
4202024-11-19T01:45:05.846ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = UnassignOximeterProducer
4212024-11-19T01:45:05.881ZINFOnexus (SagaExecutor): registered oximeter collector client file = nexus/src/app/oximeter.rs:262 id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4222024-11-19T01:45:05.885ZDEBGnexus (SagaExecutor): client request body = None method = DELETE oximeter-collector = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update uri = http://[fd00:1122:3344:101::d]:12223/producers/9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a
4232024-11-19T01:45:05.898ZDEBGnexus (SagaExecutor): client response oximeter-collector = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a result = Ok(Response { url: "http://[fd00:1122:3344:101::d]:12223/producers/9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a", status: 204, headers: {"x-request-id": "cccaa5ec-e2a1-4b58-9fff-c48ed8e0bee2", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} }) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4242024-11-19T01:45:05.898ZDEBGnexus (SagaExecutor): successfully deleted producer from collector address = [fd00:1122:3344:101::d]:12223 collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 producer_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4252024-11-19T01:45:05.899ZDEBGnexus (SEC): saga log event new_state = N005 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4262024-11-19T01:45:05.899ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 5 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4272024-11-19T01:45:05.903ZDEBGnexus (SEC): saga log event new_state = N006 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4282024-11-19T01:45:05.903ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 6 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4292024-11-19T01:45:05.906ZDEBGnexus (SagaExecutor): instance update: committing new runtime state and unlocking... instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a lock = UpdaterLock { updater_id: 9b1e1baf-58de-4735-9f10-8e295a116d78, locked_gen: Generation(Generation(3)) } new_runtime = InstanceRuntimeState { time_updated: 2024-11-19T01:45:05.659257278Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4302024-11-19T01:45:05.906ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = CommitInstanceUpdates
4312024-11-19T01:45:05.906ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = CommitInstanceUpdates
4322024-11-19T01:45:05.947ZINFOnexus (SagaExecutor): instance update: committed update new runtime state! did_unlock = true file = nexus/src/app/sagas/instance_update/mod.rs:1176 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a new_runtime = InstanceRuntimeState { time_updated: 2024-11-19T01:45:05.659257278Z, gen: Generation(Generation(4)), propolis_id: None, dst_propolis_id: None, migration_id: None, nexus_state: NoVmm, time_last_auto_restarted: None } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4332024-11-19T01:45:05.947ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Signaled
4342024-11-19T01:45:05.947ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4352024-11-19T01:45:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
4362024-11-19T01:45:05.949ZDEBGnexus (SEC): saga log event new_state = N006 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4372024-11-19T01:45:05.949ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Signaled
4382024-11-19T01:45:05.949ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4392024-11-19T01:45:05.949ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 6 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4402024-11-19T01:45:05.949ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4412024-11-19T01:45:05.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4422024-11-19T01:45:05.950ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4432024-11-19T01:45:05.952ZDEBGnexus (SEC): saga log event new_state = N007 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4442024-11-19T01:45:05.952ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 7 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4452024-11-19T01:45:05.952ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4462024-11-19T01:45:05.952ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4472024-11-19T01:45:05.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4482024-11-19T01:45:05.953ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4492024-11-19T01:45:05.955ZDEBGnexus (SEC): saga log event new_state = N007 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4502024-11-19T01:45:05.955ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Object {"instance_id": String("9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("f29067fd-e978-4a5d-8f3c-f6f062710092")}) node_id = 7 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4512024-11-19T01:45:05.956ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4522024-11-19T01:45:05.956ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4532024-11-19T01:45:05.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4542024-11-19T01:45:05.963ZDEBGnexus (SEC): saga log event new_state = N008 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4552024-11-19T01:45:05.963ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 8 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4562024-11-19T01:45:05.967ZDEBGnexus (SEC): saga log event new_state = N008 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4572024-11-19T01:45:05.967ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 8 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4582024-11-19T01:45:05.970ZDEBGnexus (SEC): saga log event new_state = N009 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4592024-11-19T01:45:05.970ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 9 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4602024-11-19T01:45:05.973ZINFOnexus (SagaExecutor): instance update (VMM destroyed): deallocating sled resource reservation file = nexus/src/app/sagas/instance_update/destroyed.rs:83 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4612024-11-19T01:45:05.973ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ReleaseSledResources
4622024-11-19T01:45:05.974ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ReleaseSledResources
4632024-11-19T01:45:05.978ZDEBGnexus (SEC): saga log event new_state = N009 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4642024-11-19T01:45:05.978ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 9 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4652024-11-19T01:45:05.981ZDEBGnexus (SEC): saga log event new_state = N010 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4662024-11-19T01:45:05.981ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 10 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4672024-11-19T01:45:05.985ZINFOnexus (SagaExecutor): instance update (VMM destroyed): marking VMM record deleted file = nexus/src/app/sagas/instance_update/destroyed.rs:114 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a propolis_id = f29067fd-e978-4a5d-8f3c-f6f062710092 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
4682024-11-19T01:45:05.985ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = MarkVmmDeleted
4692024-11-19T01:45:05.985ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = MarkVmmDeleted
4702024-11-19T01:45:05.993ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
4712024-11-19T01:45:05.993ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4722024-11-19T01:45:05.993ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a02e9a1c-f238-413c-8c59-f2a3400d349b", "content-length": "372", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
4732024-11-19T01:45:05.993ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4742024-11-19T01:45:05.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4752024-11-19T01:45:05.995ZDEBGnexus (SEC): saga log event new_state = N010 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4762024-11-19T01:45:05.995ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 10 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4772024-11-19T01:45:05.999ZDEBGnexus (SEC): saga log event new_state = N011 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4782024-11-19T01:45:05.999ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 11 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4792024-11-19T01:45:06.002ZDEBGnexus (SEC): saga log event new_state = N011 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4802024-11-19T01:45:06.002ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 11 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4812024-11-19T01:45:06.006ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4822024-11-19T01:45:06.006ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4832024-11-19T01:45:06.006ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8331de0f-7217-4e2f-b2a2-b7d430146959", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
4842024-11-19T01:45:06.006ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4852024-11-19T01:45:06.006ZDEBGnexus (SEC): saga log event new_state = N012 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
4862024-11-19T01:45:06.006ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 12 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
4872024-11-19T01:45:06.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4882024-11-19T01:45:06.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4892024-11-19T01:45:06.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4902024-11-19T01:45:06.009ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4912024-11-19T01:45:06.009ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4922024-11-19T01:45:06.011ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4932024-11-19T01:45:06.011ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4942024-11-19T01:45:06.011ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4952024-11-19T01:45:06.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4962024-11-19T01:45:06.012ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4972024-11-19T01:45:06.012ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4982024-11-19T01:45:06.014ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
4992024-11-19T01:45:06.015ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
5002024-11-19T01:45:06.015ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
5012024-11-19T01:45:06.015ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2c42e300-0adc-4cfc-9357-f4aeefba8fd1", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
5022024-11-19T01:45:06.016ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 67.634959ms iteration = 28 status = {}
5032024-11-19T01:45:06.017ZDEBGnexus (SagaExecutor): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Instance { parent: Project { parent: Silo { parent: Fleet, key: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, lookup_type: ById(7ae1255b-c149-4b2a-8f4e-7489cdeaaefa) }, key: 43ed357a-1904-4080-ab4e-07192398e688, lookup_type: ById(43ed357a-1904-4080-ab4e-07192398e688) }, key: 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a, lookup_type: ById(9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a) } result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
5042024-11-19T01:45:06.017ZDEBGnexus (SagaExecutor): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true roles = RoleSet { roles: {} } saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
5052024-11-19T01:45:06.017ZDEBGnexus (SagaExecutor): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true resource = Database result = Ok(()) saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_node = ChainSuccessorSaga
5062024-11-19T01:45:06.020ZDEBGnexus (SagaExecutor): instance update: instance will not reincarnate auto_restart_config = InstanceAutoRestart { policy: None, cooldown: None } instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a karmic_state = WillReincarnate needs_reincarnation = false saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update
5072024-11-19T01:45:06.020ZDEBGnexus (SEC): saga log event new_state = N012 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5082024-11-19T01:45:06.020ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 12 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
5092024-11-19T01:45:06.024ZDEBGnexus (SEC): saga log event new_state = N014 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5102024-11-19T01:45:06.024ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 14 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
5112024-11-19T01:45:06.028ZDEBGnexus (SEC): saga log event new_state = N014 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5122024-11-19T01:45:06.028ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 14 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
5132024-11-19T01:45:06.031ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5142024-11-19T01:45:06.031ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143
5152024-11-19T01:45:06.046ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5162024-11-19T01:45:06.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5172024-11-19T01:45:06.049ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 02480ad3-1b93-4e52-87e4-2f6932e70741 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
5182024-11-19T01:45:06.050ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 02480ad3-1b93-4e52-87e4-2f6932e70741 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
5192024-11-19T01:45:06.052ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2670 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 02480ad3-1b93-4e52-87e4-2f6932e70741 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
5202024-11-19T01:45:06.068ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5212024-11-19T01:45:06.068ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 saga_name = instance-update saga_result = SagaResult { saga_id: 08c9fd59-8b70-486c-b331-0633c35ea143, saga_log: SagaLog { saga_id: 08c9fd59-8b70-486c-b331-0633c35ea143, unwinding: false, events: [N013 started, N013 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N009 started, N009 succeeded, N010 started, N010 succeeded, N011 started, N011 succeeded, N012 started, N012 succeeded, N014 started, N014 succeeded], node_status: {0: Succeeded(String("9b1e1baf-58de-4735-9f10-8e295a116d78")), 1: Succeeded(Object {"locked_gen": Number(3), "updater_id": String("9b1e1baf-58de-4735-9f10-8e295a116d78")}), 2: Succeeded(String("Delete")), 3: Succeeded(Null), 4: Succeeded(Null), 5: Succeeded(Null), 6: Succeeded(Null), 7: Succeeded(Object {"instance_id": String("9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("f29067fd-e978-4a5d-8f3c-f6f062710092")}), 8: Succeeded(Null), 9: Succeeded(Null), 10: Succeeded(Null), 11: Succeeded(Null), 12: Succeeded(Null), 13: Succeeded(Null), 14: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"active_vmm_destroyed": Null, "chain_successor_saga": Null, "commit_instance_updates": Null, "destroy_active_vmm_params": Object {"instance_id": String("9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a"), "serialized_authn": Object {"kind": Object {"Authenticated": Array [Object {"actor": Object {"UserBuiltin": Object {"user_builtin_id": String("001de000-05e4-4000-8000-000000000002")}}}, Null]}}, "vmm_id": String("f29067fd-e978-4a5d-8f3c-f6f062710092")}, "mark_vmm_deleted": Null, "network_config_update": String("Delete"), "release_sled_resources": Null, "release_virtual_provisioning": Null, "saga_instance_lock_id": String("9b1e1baf-58de-4735-9f10-8e295a116d78"), "unassign_oximeter_producer": Null, "update_network_config": Null, "updater_lock": Object {"locked_gen": Number(3), "updater_id": String("9b1e1baf-58de-4735-9f10-8e295a116d78")}} }) }
5222024-11-19T01:45:06.068ZDEBGnexus (SagaExecutor): instance update: child saga completed successfully child_saga_id = 08c9fd59-8b70-486c-b331-0633c35ea143 instance_id = 9dbe2cf6-3afc-4bd1-9876-14136d7f2e8a saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update
5232024-11-19T01:45:06.068ZDEBGnexus (SEC): saga log event new_state = N002 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5242024-11-19T01:45:06.068ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 2 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
5252024-11-19T01:45:06.072ZDEBGnexus (SEC): saga log event new_state = N004 started sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5262024-11-19T01:45:06.072ZDEBGnexus (SecStore): recording saga event event_type = Started node_id = 4 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
5272024-11-19T01:45:06.075ZDEBGnexus (SEC): saga log event new_state = N004 succeeded sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5282024-11-19T01:45:06.075ZDEBGnexus (SecStore): recording saga event event_type = Succeeded(Null) node_id = 4 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
5292024-11-19T01:45:06.078ZINFOnexus (SEC): update for saga cached state file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1384 new_state = Done saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5302024-11-19T01:45:06.078ZINFOnexus (SecStore): updating state file = nexus/db-queries/src/db/sec_store.rs:95 new_state = done saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691
5312024-11-19T01:45:06.082ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5322024-11-19T01:45:06.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5332024-11-19T01:45:06.084ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5342024-11-19T01:45:06.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3b1e161a-7336-4f3c-9869-cfda8c2b2e91 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5352024-11-19T01:45:06.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3b1e161a-7336-4f3c-9869-cfda8c2b2e91 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5362024-11-19T01:45:06.085ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5372024-11-19T01:45:06.085ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5382024-11-19T01:45:06.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5392024-11-19T01:45:06.087ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2195 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3b1e161a-7336-4f3c-9869-cfda8c2b2e91 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5402024-11-19T01:45:06.087ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5412024-11-19T01:45:06.088ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
5422024-11-19T01:45:06.088ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5432024-11-19T01:45:06.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5442024-11-19T01:45:06.112ZINFOnexus (SEC): saga finished file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/steno-0.4.1/src/sec.rs:1025 result = success saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update sec_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5452024-11-19T01:45:06.112ZINFOnexus (SagaExecutor): saga finished file = nexus/src/app/saga.rs:404 saga_id = 5d81abd8-5694-442e-b73d-d5fb9d096691 saga_name = start-instance-update saga_result = SagaResult { saga_id: 5d81abd8-5694-442e-b73d-d5fb9d096691, saga_log: SagaLog { saga_id: 5d81abd8-5694-442e-b73d-d5fb9d096691, unwinding: false, events: [N003 started, N003 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N004 started, N004 succeeded], node_status: {0: Succeeded(String("283e2008-df56-4825-ae30-ae75cee4007f")), 1: Succeeded(Object {"locked_gen": Number(2), "updater_id": String("283e2008-df56-4825-ae30-ae75cee4007f")}), 2: Succeeded(Null), 3: Succeeded(Null), 4: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Null, node_outputs: {"saga_instance_lock_id": String("283e2008-df56-4825-ae30-ae75cee4007f"), "state": Null, "updater_lock": Object {"locked_gen": Number(2), "updater_id": String("283e2008-df56-4825-ae30-ae75cee4007f")}} }) }
5462024-11-19T01:45:06.121ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
5472024-11-19T01:45:06.121ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e4c567e-0f94-48ca-ab4c-4f99429a71e8", "content-length": "691", "date": "Tue, 19 Nov 2024 01:45:05 GMT"} })
5482024-11-19T01:45:06.121ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
5492024-11-19T01:45:06.121ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
5502024-11-19T01:45:06.121ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
5512024-11-19T01:45:06.123ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 176.562877ms iteration = 27 status = {}
5522024-11-19T01:45:07.109ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 06384a76-c809-411c-9d84-32b600e6cd4e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5532024-11-19T01:45:07.109ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 06384a76-c809-411c-9d84-32b600e6cd4e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5542024-11-19T01:45:07.110ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1806 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 06384a76-c809-411c-9d84-32b600e6cd4e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5552024-11-19T01:45:08.134ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd584c5f-e8a9-4ca8-95cd-3a52616b144b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5562024-11-19T01:45:08.134ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd584c5f-e8a9-4ca8-95cd-3a52616b144b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5572024-11-19T01:45:08.137ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2750 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd584c5f-e8a9-4ca8-95cd-3a52616b144b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5582024-11-19T01:45:08.389ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
5592024-11-19T01:45:08.389ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
5602024-11-19T01:45:08.419ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
5612024-11-19T01:45:08.419ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:64114
5622024-11-19T01:45:08.419ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:64114 req_id = b14b7f9e-285b-4619-bfca-bbae9b4150b0 roles = RoleSet { roles: {} } uri = /metrics/producers
5632024-11-19T01:45:08.420ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:64114 req_id = b14b7f9e-285b-4619-bfca-bbae9b4150b0 resource = Database result = Ok(()) uri = /metrics/producers
5642024-11-19T01:45:08.459ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
5652024-11-19T01:45:08.459ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39911 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:64114 req_id = b14b7f9e-285b-4619-bfca-bbae9b4150b0 response_code = 201 uri = /metrics/producers
5662024-11-19T01:45:08.459ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "b14b7f9e-285b-4619-bfca-bbae9b4150b0", "content-length": "41", "date": "Tue, 19 Nov 2024 01:45:08 GMT"} })
5672024-11-19T01:45:08.461ZDEBGnexus (producer-registration-task): registered with nexus successfully
5682024-11-19T01:45:08.461ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
5692024-11-19T01:45:09.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a588f5e9-4196-4046-ad5b-e0610f00ed23 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5702024-11-19T01:45:09.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a588f5e9-4196-4046-ad5b-e0610f00ed23 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5712024-11-19T01:45:09.162ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2689 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a588f5e9-4196-4046-ad5b-e0610f00ed23 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5722024-11-19T01:45:10.185ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c720b2b4-866b-4d92-b256-a33528b847a9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5732024-11-19T01:45:10.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c720b2b4-866b-4d92-b256-a33528b847a9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5742024-11-19T01:45:10.187ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2425 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c720b2b4-866b-4d92-b256-a33528b847a9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5752024-11-19T01:45:11.210ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d1774b65-531e-482e-ba6a-7196621c4108 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5762024-11-19T01:45:11.210ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d1774b65-531e-482e-ba6a-7196621c4108 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5772024-11-19T01:45:11.212ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2032 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d1774b65-531e-482e-ba6a-7196621c4108 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5782024-11-19T01:45:11.809ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1458 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = ed3fc3a1-149e-422a-b46c-83602f832b4e response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
5792024-11-19T01:45:12.233ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 304a8006-6197-49dc-a7d2-78a83c2b37b0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5802024-11-19T01:45:12.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 304a8006-6197-49dc-a7d2-78a83c2b37b0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5812024-11-19T01:45:12.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2534 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 304a8006-6197-49dc-a7d2-78a83c2b37b0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5822024-11-19T01:45:13.257ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c69b50e-31cb-4cc2-b81e-353ad7e5231c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5832024-11-19T01:45:13.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c69b50e-31cb-4cc2-b81e-353ad7e5231c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5842024-11-19T01:45:13.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2234 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c69b50e-31cb-4cc2-b81e-353ad7e5231c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
5852024-11-19T01:45:13.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
5862024-11-19T01:45:13.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5872024-11-19T01:45:13.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5882024-11-19T01:45:13.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
5892024-11-19T01:45:13.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5902024-11-19T01:45:13.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5912024-11-19T01:45:13.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5922024-11-19T01:45:13.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5932024-11-19T01:45:13.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5942024-11-19T01:45:13.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5952024-11-19T01:45:13.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5962024-11-19T01:45:13.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
5972024-11-19T01:45:13.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982024-11-19T01:45:13.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
5992024-11-19T01:45:13.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6002024-11-19T01:45:13.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6012024-11-19T01:45:13.551ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
6022024-11-19T01:45:13.551ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.486074ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
6032024-11-19T01:45:13.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6042024-11-19T01:45:13.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6052024-11-19T01:45:13.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.347311ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6062024-11-19T01:45:14.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 60ee3851-7403-413f-b35b-4efcb7923fbb roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6072024-11-19T01:45:14.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 60ee3851-7403-413f-b35b-4efcb7923fbb resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6082024-11-19T01:45:14.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2076 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 60ee3851-7403-413f-b35b-4efcb7923fbb response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6092024-11-19T01:45:15.305ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4dcae369-d4a2-4e56-b302-a2a79164f5b8 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6102024-11-19T01:45:15.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4dcae369-d4a2-4e56-b302-a2a79164f5b8 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6112024-11-19T01:45:15.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1893 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4dcae369-d4a2-4e56-b302-a2a79164f5b8 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6122024-11-19T01:45:16.208ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:36829
6132024-11-19T01:45:16.209ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:36829 req_id = aca09b3b-6bf5-4593-b350-7af05f834adf roles = RoleSet { roles: {} } uri = /metrics/producers
6142024-11-19T01:45:16.209ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:36829 req_id = aca09b3b-6bf5-4593-b350-7af05f834adf resource = Database result = Ok(()) uri = /metrics/producers
6152024-11-19T01:45:16.249ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 1b898a1c-dd6e-4a5f-b7b4-a7823ab855c5
6162024-11-19T01:45:16.249ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40406 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:36829 req_id = aca09b3b-6bf5-4593-b350-7af05f834adf response_code = 201 uri = /metrics/producers
6172024-11-19T01:45:16.328ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f1de040e-9962-425f-b58c-6a92db4f6b02 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6182024-11-19T01:45:16.329ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f1de040e-9962-425f-b58c-6a92db4f6b02 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6192024-11-19T01:45:16.331ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2484 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f1de040e-9962-425f-b58c-6a92db4f6b02 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6202024-11-19T01:45:17.353ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d3e563a-1a13-46ca-9aca-7ab081b68970 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6212024-11-19T01:45:17.353ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d3e563a-1a13-46ca-9aca-7ab081b68970 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6222024-11-19T01:45:17.355ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2284 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d3e563a-1a13-46ca-9aca-7ab081b68970 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6232024-11-19T01:45:18.376ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2b430f48-afb1-4784-a0f3-3cd27fb0679a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6242024-11-19T01:45:18.377ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2b430f48-afb1-4784-a0f3-3cd27fb0679a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6252024-11-19T01:45:18.379ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2594 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2b430f48-afb1-4784-a0f3-3cd27fb0679a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6262024-11-19T01:45:19.400ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7b97a177-df00-4764-9f71-b4b1ca1a7d82 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6272024-11-19T01:45:19.401ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7b97a177-df00-4764-9f71-b4b1ca1a7d82 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6282024-11-19T01:45:19.402ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1904 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7b97a177-df00-4764-9f71-b4b1ca1a7d82 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6292024-11-19T01:45:20.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e2484db9-8dbc-439c-af3f-c35e424521eb roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6302024-11-19T01:45:20.424ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e2484db9-8dbc-439c-af3f-c35e424521eb resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6312024-11-19T01:45:20.426ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e2484db9-8dbc-439c-af3f-c35e424521eb response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6322024-11-19T01:45:21.447ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d9bc0a24-95fb-48d2-9e6f-55a4b05302eb roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6332024-11-19T01:45:21.448ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d9bc0a24-95fb-48d2-9e6f-55a4b05302eb resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6342024-11-19T01:45:21.450ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2507 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d9bc0a24-95fb-48d2-9e6f-55a4b05302eb response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6352024-11-19T01:45:22.471ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e880fcb7-eec1-4ee8-8b92-639069f86d0c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6362024-11-19T01:45:22.471ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e880fcb7-eec1-4ee8-8b92-639069f86d0c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6372024-11-19T01:45:22.473ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2503 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e880fcb7-eec1-4ee8-8b92-639069f86d0c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6382024-11-19T01:45:23.495ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7a8c6b86-0070-4ece-b5ce-419f3205f20b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6392024-11-19T01:45:23.495ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7a8c6b86-0070-4ece-b5ce-419f3205f20b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6402024-11-19T01:45:23.497ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1819 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7a8c6b86-0070-4ece-b5ce-419f3205f20b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6412024-11-19T01:45:23.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
6422024-11-19T01:45:23.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6432024-11-19T01:45:23.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6442024-11-19T01:45:23.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6452024-11-19T01:45:23.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
6462024-11-19T01:45:23.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6472024-11-19T01:45:23.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6482024-11-19T01:45:23.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6492024-11-19T01:45:23.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6502024-11-19T01:45:23.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6512024-11-19T01:45:23.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
6522024-11-19T01:45:23.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6532024-11-19T01:45:23.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6542024-11-19T01:45:23.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6552024-11-19T01:45:23.535ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.730597ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6562024-11-19T01:45:23.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6572024-11-19T01:45:23.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
6582024-11-19T01:45:23.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6592024-11-19T01:45:23.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6602024-11-19T01:45:23.551ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
6612024-11-19T01:45:23.551ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.706158ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
6622024-11-19T01:45:24.519ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8119185e-e86c-4564-bcbe-71764e837ffd roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6632024-11-19T01:45:24.519ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8119185e-e86c-4564-bcbe-71764e837ffd resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6642024-11-19T01:45:24.521ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2461 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8119185e-e86c-4564-bcbe-71764e837ffd response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6652024-11-19T01:45:25.543ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 27292083-5ddb-4dc6-8843-0bee08bf070c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6662024-11-19T01:45:25.543ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 27292083-5ddb-4dc6-8843-0bee08bf070c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6672024-11-19T01:45:25.545ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2303 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 27292083-5ddb-4dc6-8843-0bee08bf070c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6682024-11-19T01:45:26.568ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05e1e9d2-5ab5-45f9-800c-f3ea9183ec82 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6692024-11-19T01:45:26.568ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05e1e9d2-5ab5-45f9-800c-f3ea9183ec82 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6702024-11-19T01:45:26.570ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2374 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05e1e9d2-5ab5-45f9-800c-f3ea9183ec82 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6712024-11-19T01:45:26.810ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1714 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 156184b0-4a31-4a6c-ab48-9a217dcf3527 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
6722024-11-19T01:45:26.902ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:52368
6732024-11-19T01:45:26.902ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52368 req_id = 95c79b30-cc13-4b0b-892f-c8bf6be72fca roles = RoleSet { roles: {} } uri = /metrics/producers
6742024-11-19T01:45:26.902ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52368 req_id = 95c79b30-cc13-4b0b-892f-c8bf6be72fca resource = Database result = Ok(()) uri = /metrics/producers
6752024-11-19T01:45:26.939ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = e791ed89-1335-4e80-9045-8f1c5065489f
6762024-11-19T01:45:26.939ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37152 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:52368 req_id = 95c79b30-cc13-4b0b-892f-c8bf6be72fca response_code = 201 uri = /metrics/producers
6772024-11-19T01:45:27.592ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d88fdb6-c43f-4017-bcbc-f180476bdc33 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6782024-11-19T01:45:27.592ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d88fdb6-c43f-4017-bcbc-f180476bdc33 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6792024-11-19T01:45:27.594ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2083 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d88fdb6-c43f-4017-bcbc-f180476bdc33 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6802024-11-19T01:45:28.615ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = cd26dcaf-e500-42c2-bbb7-61b4c86617a2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6812024-11-19T01:45:28.616ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = cd26dcaf-e500-42c2-bbb7-61b4c86617a2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6822024-11-19T01:45:28.618ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2504 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = cd26dcaf-e500-42c2-bbb7-61b4c86617a2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6832024-11-19T01:45:29.640ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ee7c39c4-a65c-4511-a17e-2546bc679a8d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6842024-11-19T01:45:29.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ee7c39c4-a65c-4511-a17e-2546bc679a8d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6852024-11-19T01:45:29.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2626 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ee7c39c4-a65c-4511-a17e-2546bc679a8d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6862024-11-19T01:45:30.664ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 772c9cca-3f1d-4a1a-92aa-fa99acf2293a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6872024-11-19T01:45:30.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 772c9cca-3f1d-4a1a-92aa-fa99acf2293a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6882024-11-19T01:45:30.666ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2483 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 772c9cca-3f1d-4a1a-92aa-fa99acf2293a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6892024-11-19T01:45:31.687ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = db5758e5-d29d-4fa7-9a02-ce75380ed3fa roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6902024-11-19T01:45:31.687ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = db5758e5-d29d-4fa7-9a02-ce75380ed3fa resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6912024-11-19T01:45:31.689ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2625 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = db5758e5-d29d-4fa7-9a02-ce75380ed3fa response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6922024-11-19T01:45:31.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = bfc3a1e5-ab1a-4e7b-90c5-75768b3604ff roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
6932024-11-19T01:45:31.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = bfc3a1e5-ab1a-4e7b-90c5-75768b3604ff resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
6942024-11-19T01:45:31.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3262 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = bfc3a1e5-ab1a-4e7b-90c5-75768b3604ff response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
6952024-11-19T01:45:31.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 4155dd35-5988-4add-bdf3-37268ca7d9a5 roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
6962024-11-19T01:45:31.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 4155dd35-5988-4add-bdf3-37268ca7d9a5 resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
6972024-11-19T01:45:31.807ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2729 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 4155dd35-5988-4add-bdf3-37268ca7d9a5 response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
6982024-11-19T01:45:32.710ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b1ae6d12-753b-4d8d-b619-8348ff9515ce roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
6992024-11-19T01:45:32.710ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b1ae6d12-753b-4d8d-b619-8348ff9515ce resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
7002024-11-19T01:45:32.712ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2233 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b1ae6d12-753b-4d8d-b619-8348ff9515ce response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
7012024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
7022024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 15 reason = Timeout
7032024-11-19T01:45:33.529ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-19 01:35:33.529768631 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
7042024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
7052024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
7062024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 15 reason = Timeout
7072024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
7082024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 15 reason = Timeout
7092024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
7102024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
7112024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
7122024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
7132024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
7142024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 15 reason = Timeout
7152024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
7162024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
7172024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
7182024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 27 reason = Timeout
7192024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
7202024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 15 reason = Timeout
7212024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
7222024-11-19T01:45:33.529ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
7232024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
7242024-11-19T01:45:33.530ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120
7252024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.45\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120"}
7262024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
7272024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
7282024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
7292024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
7302024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7312024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
7322024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
7332024-11-19T01:45:33.530ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("52e4d19a-b928-4138-8817-4fd8deeaf29e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 697625834 } })
7342024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
7352024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7362024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
7372024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7382024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
7392024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7402024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
7412024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
7422024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7432024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7442024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
7452024-11-19T01:45:33.530ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
7462024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
7472024-11-19T01:45:33.530ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e98d393-d940-4def-ba7c-d8c67ea23aa1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e48e23d9-6a0a-4bc6-9e0b-f83ba78952f1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 697805728 } })
7482024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7492024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7502024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7512024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7522024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7532024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
7542024-11-19T01:45:33.530ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.174011ms iteration = 15 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
7552024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
7562024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
7572024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.299603ms iteration = 15 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
7582024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Timeout
7592024-11-19T01:45:33.531ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
7602024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7612024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
7622024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
7632024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7642024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7652024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
7662024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7672024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7682024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
7692024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7702024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7712024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7722024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7732024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7742024-11-19T01:45:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7752024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7762024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
7772024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
7782024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7792024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7802024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
7812024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7822024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
7832024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7842024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
7852024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7862024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
7872024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7882024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7892024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7902024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
7912024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
7922024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7932024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.107735ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
7942024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7952024-11-19T01:45:33.532ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-19 01:35:33.529768631 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
7962024-11-19T01:45:33.532ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.227178ms iteration = 14 status = {"expiration":"2024-11-19T01:35:33.529768631Z","pruned":[]}
7972024-11-19T01:45:33.533ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 700532106 } })
7982024-11-19T01:45:33.533ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
7992024-11-19T01:45:33.534ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8002024-11-19T01:45:33.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8012024-11-19T01:45:33.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8022024-11-19T01:45:33.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8032024-11-19T01:45:33.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8042024-11-19T01:45:33.538ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
8052024-11-19T01:45:33.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
8062024-11-19T01:45:33.540ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 708076421 } })
8072024-11-19T01:45:33.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8082024-11-19T01:45:33.541ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
8092024-11-19T01:45:33.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8102024-11-19T01:45:33.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8112024-11-19T01:45:33.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8122024-11-19T01:45:33.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8132024-11-19T01:45:33.544ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8142024-11-19T01:45:33.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8152024-11-19T01:45:33.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8162024-11-19T01:45:33.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8172024-11-19T01:45:33.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8182024-11-19T01:45:33.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8192024-11-19T01:45:33.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8202024-11-19T01:45:33.549ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8212024-11-19T01:45:33.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
8222024-11-19T01:45:33.552ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
8232024-11-19T01:45:33.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
8242024-11-19T01:45:33.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8252024-11-19T01:45:33.556ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 24.00713ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
8262024-11-19T01:45:33.557ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8272024-11-19T01:45:33.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8282024-11-19T01:45:33.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8292024-11-19T01:45:33.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8302024-11-19T01:45:33.560ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8312024-11-19T01:45:33.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8322024-11-19T01:45:33.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8332024-11-19T01:45:33.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8342024-11-19T01:45:33.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
8352024-11-19T01:45:33.569ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
8362024-11-19T01:45:33.569ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
8372024-11-19T01:45:33.569ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
8382024-11-19T01:45:33.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8392024-11-19T01:45:33.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8402024-11-19T01:45:33.571ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8412024-11-19T01:45:33.571ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1783, tv_nsec: 738708149 } })
8422024-11-19T01:45:33.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8432024-11-19T01:45:33.571ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
8442024-11-19T01:45:33.572ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8452024-11-19T01:45:33.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
8462024-11-19T01:45:33.574ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8472024-11-19T01:45:33.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8482024-11-19T01:45:33.576ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8492024-11-19T01:45:33.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8502024-11-19T01:45:33.578ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8512024-11-19T01:45:33.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8522024-11-19T01:45:33.580ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8532024-11-19T01:45:33.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8542024-11-19T01:45:33.586ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8552024-11-19T01:45:33.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8562024-11-19T01:45:33.588ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8572024-11-19T01:45:33.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
8582024-11-19T01:45:33.590ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8592024-11-19T01:45:33.590ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8602024-11-19T01:45:33.590ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8612024-11-19T01:45:33.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8622024-11-19T01:45:33.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8632024-11-19T01:45:33.591ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8642024-11-19T01:45:33.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8652024-11-19T01:45:33.593ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8662024-11-19T01:45:33.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8672024-11-19T01:45:33.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
8682024-11-19T01:45:33.595ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8692024-11-19T01:45:33.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8702024-11-19T01:45:33.603ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
8712024-11-19T01:45:33.604ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
8722024-11-19T01:45:33.604ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
8732024-11-19T01:45:33.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
8742024-11-19T01:45:33.604ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 72.202651ms iteration = 27 status = {"errors":[],"records_set_to_done":[]}
8752024-11-19T01:45:33.609ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
8762024-11-19T01:45:33.609ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.732362ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
8772024-11-19T01:45:33.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8782024-11-19T01:45:33.610ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
8792024-11-19T01:45:33.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
8802024-11-19T01:45:33.616ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8812024-11-19T01:45:33.617ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
8822024-11-19T01:45:33.617ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
8832024-11-19T01:45:33.617ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 84.507241ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
8842024-11-19T01:45:33.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8852024-11-19T01:45:33.618ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8862024-11-19T01:45:33.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8872024-11-19T01:45:33.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8882024-11-19T01:45:33.624ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
8892024-11-19T01:45:33.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
8902024-11-19T01:45:33.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8912024-11-19T01:45:33.626ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
8922024-11-19T01:45:33.626ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
8932024-11-19T01:45:33.626ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 95.485817ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
8942024-11-19T01:45:33.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8952024-11-19T01:45:33.629ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
8962024-11-19T01:45:33.629ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
8972024-11-19T01:45:33.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8982024-11-19T01:45:33.629ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1befe38-9a09-4d99-a8d7-c9bff2db9bb2", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
8992024-11-19T01:45:33.629ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9002024-11-19T01:45:33.631ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9012024-11-19T01:45:33.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
9022024-11-19T01:45:33.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9032024-11-19T01:45:33.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9042024-11-19T01:45:33.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9052024-11-19T01:45:33.633ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9062024-11-19T01:45:33.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9072024-11-19T01:45:33.641ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
9082024-11-19T01:45:33.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
9092024-11-19T01:45:33.644ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
9102024-11-19T01:45:33.644ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
9112024-11-19T01:45:33.644ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 115.096499ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
9122024-11-19T01:45:33.645ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 114.935855ms iteration = 31 status = {"physical_disks_added":0}
9132024-11-19T01:45:33.645ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9142024-11-19T01:45:33.645ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
9152024-11-19T01:45:33.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 115.337082ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
9162024-11-19T01:45:33.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9172024-11-19T01:45:33.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9182024-11-19T01:45:33.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9192024-11-19T01:45:33.650ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9202024-11-19T01:45:33.650ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
9212024-11-19T01:45:33.650ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9222024-11-19T01:45:33.651ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b67b2ffc-78e4-41f8-a8d8-e9a32b201de0", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
9232024-11-19T01:45:33.651ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9242024-11-19T01:45:33.652ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
9252024-11-19T01:45:33.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 122.332388ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9262024-11-19T01:45:33.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
9272024-11-19T01:45:33.654ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
9282024-11-19T01:45:33.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
9292024-11-19T01:45:33.654ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9302024-11-19T01:45:33.654ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
9312024-11-19T01:45:33.654ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9322024-11-19T01:45:33.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9332024-11-19T01:45:33.656ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 126.020674ms iteration = 27 status = 0
9342024-11-19T01:45:33.658ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
9352024-11-19T01:45:33.658ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 126.635375ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
9362024-11-19T01:45:33.659ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 126.91914ms iteration = 27 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
9372024-11-19T01:45:33.661ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-19T01:32:28.835276Z, time_modified: 2024-11-19T01:32:28.835276Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
9382024-11-19T01:45:33.661ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 131.360639ms iteration = 15 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]}},"default_endpoint":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
9392024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9402024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
9412024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
9422024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
9432024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9442024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9452024-11-19T01:45:33.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9462024-11-19T01:45:33.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9472024-11-19T01:45:33.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
9482024-11-19T01:45:33.670ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9492024-11-19T01:45:33.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9502024-11-19T01:45:33.676ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
9512024-11-19T01:45:33.676ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
9522024-11-19T01:45:33.676ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c3e2722-622f-4165-8e87-75346efe0954", "content-length": "26", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
9532024-11-19T01:45:33.676ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
9542024-11-19T01:45:33.677ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
9552024-11-19T01:45:33.677ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 0 version = 2
9562024-11-19T01:45:33.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9572024-11-19T01:45:33.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9582024-11-19T01:45:33.679ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
9592024-11-19T01:45:33.679ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9602024-11-19T01:45:33.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
9612024-11-19T01:45:33.682ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9622024-11-19T01:45:33.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9632024-11-19T01:45:33.684ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 1 version = 2
9642024-11-19T01:45:33.684ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
9652024-11-19T01:45:33.684ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-19 01:32:36.054328 UTC dns_group = external generation = 2
9662024-11-19T01:45:33.684ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 154.969881ms iteration = 15 status = {"generation":2}
9672024-11-19T01:45:33.697ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9682024-11-19T01:45:33.697ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
9692024-11-19T01:45:33.697ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
9702024-11-19T01:45:33.697ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 166.451767ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
9712024-11-19T01:45:33.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9722024-11-19T01:45:33.702ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 169.795457ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
9732024-11-19T01:45:33.703ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
9742024-11-19T01:45:33.703ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
9752024-11-19T01:45:33.703ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9762024-11-19T01:45:33.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9772024-11-19T01:45:33.712ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
9782024-11-19T01:45:33.712ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 180.888264ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
9792024-11-19T01:45:33.734ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05629a7f-7adc-46b8-a466-7213bdd05c6f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
9802024-11-19T01:45:33.734ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05629a7f-7adc-46b8-a466-7213bdd05c6f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
9812024-11-19T01:45:33.789ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
9822024-11-19T01:45:33.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
9832024-11-19T01:45:33.801ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
9842024-11-19T01:45:33.801ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa03f9c6-87ac-4a81-bdd0-f5de44b915b7", "content-length": "2", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
9852024-11-19T01:45:33.802ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
9862024-11-19T01:45:33.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
9872024-11-19T01:45:33.823ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
9882024-11-19T01:45:33.838ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 308.097919ms iteration = 27 status = {}
9892024-11-19T01:45:33.840ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
9902024-11-19T01:45:33.840ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
9912024-11-19T01:45:33.840ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f99e72fb-2dab-4f18-9d64-65acfaf93391", "content-length": "372", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
9922024-11-19T01:45:33.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
9932024-11-19T01:45:33.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
9942024-11-19T01:45:33.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9952024-11-19T01:45:33.842ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108179 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05629a7f-7adc-46b8-a466-7213bdd05c6f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
9962024-11-19T01:45:33.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9972024-11-19T01:45:33.844ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 314.163518ms iteration = 27 status = null
9982024-11-19T01:45:33.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
9992024-11-19T01:45:33.845ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10002024-11-19T01:45:33.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10012024-11-19T01:45:33.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10022024-11-19T01:45:33.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10032024-11-19T01:45:33.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10042024-11-19T01:45:33.848ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10052024-11-19T01:45:33.849ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
10062024-11-19T01:45:33.849ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
10072024-11-19T01:45:33.849ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-19 01:32:33.664524 UTC dns_group = internal generation = 1
10082024-11-19T01:45:33.849ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 319.697736ms iteration = 15 status = {"generation":1}
10092024-11-19T01:45:33.850ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
10102024-11-19T01:45:33.850ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10112024-11-19T01:45:33.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
10122024-11-19T01:45:33.864ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
10132024-11-19T01:45:33.864ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
10142024-11-19T01:45:33.864ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "278275c7-0090-4942-b3bb-c4eddf76439f", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
10152024-11-19T01:45:33.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10162024-11-19T01:45:33.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10172024-11-19T01:45:33.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10182024-11-19T01:45:33.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10192024-11-19T01:45:33.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10202024-11-19T01:45:33.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10212024-11-19T01:45:33.874ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
10222024-11-19T01:45:33.874ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
10232024-11-19T01:45:33.874ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "51a9094e-316d-43bb-ac0e-b076e6d73840", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
10242024-11-19T01:45:33.875ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 344.457169ms iteration = 29 status = {}
10252024-11-19T01:45:33.893ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
10262024-11-19T01:45:33.894ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "44babeb6-d883-49da-b56f-667d7b0b9473", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10272024-11-19T01:45:33.894ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10282024-11-19T01:45:33.896ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "80143506-92fb-41ff-8760-865d270de141", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10292024-11-19T01:45:33.896ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10302024-11-19T01:45:33.897ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
10312024-11-19T01:45:33.897ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 367.705834ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
10322024-11-19T01:45:33.917ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
10332024-11-19T01:45:33.918ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "4b9a570c-4a90-4e02-8aab-2402a195eee3", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10342024-11-19T01:45:33.918ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10352024-11-19T01:45:33.919ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "8c46bd18-a287-4815-97df-446cf0214791", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10362024-11-19T01:45:33.919ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10372024-11-19T01:45:33.920ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "affc6fc8-7ea9-4b24-99a3-cb1dde624fbe", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10382024-11-19T01:45:33.920ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10392024-11-19T01:45:33.922ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
10402024-11-19T01:45:33.922ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 392.741072ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
10412024-11-19T01:45:34.049ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
10422024-11-19T01:45:34.069ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10432024-11-19T01:45:34.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10442024-11-19T01:45:34.072ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
10452024-11-19T01:45:34.072ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10462024-11-19T01:45:34.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10472024-11-19T01:45:34.074ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "00fc4a23-1c02-4ca6-bedf-e28e6ecd9961", "content-length": "691", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
10482024-11-19T01:45:34.074ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
10492024-11-19T01:45:34.074ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
10502024-11-19T01:45:34.074ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
10512024-11-19T01:45:34.076ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 545.874852ms iteration = 28 status = {}
10522024-11-19T01:45:34.080ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
10532024-11-19T01:45:34.080ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc3758b1-5e7d-4001-9e9f-9d38e7108df9", "content-length": "134", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
10542024-11-19T01:45:34.080ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
10552024-11-19T01:45:34.080ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
10562024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
10572024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
10582024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10592024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10602024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10612024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10622024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
10632024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
10642024-11-19T01:45:34.081ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
10652024-11-19T01:45:34.081ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10662024-11-19T01:45:34.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10672024-11-19T01:45:34.083ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
10682024-11-19T01:45:34.083ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "20fafb84-e0d8-44f7-8472-eadb256c1c73", "content-length": "2", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
10692024-11-19T01:45:34.083ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
10702024-11-19T01:45:34.084ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad189592-76d5-4d27-a1ee-f9dab758e646", "content-length": "51", "date": "Tue, 19 Nov 2024 01:45:33 GMT"} })
10712024-11-19T01:45:34.084ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
10722024-11-19T01:45:34.084ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
10732024-11-19T01:45:34.186ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e59ab170-14e9-4b21-8ba9-f99f964cfe93", "date": "Tue, 19 Nov 2024 01:45:34 GMT"} })
10742024-11-19T01:45:34.186ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
10752024-11-19T01:45:34.186ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
10762024-11-19T01:45:34.186ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e905692-bb15-4a6a-89c7-42ea93800dc9", "content-length": "682", "date": "Tue, 19 Nov 2024 01:45:34 GMT"} })
10772024-11-19T01:45:34.186ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10782024-11-19T01:45:34.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10792024-11-19T01:45:34.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10802024-11-19T01:45:34.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10812024-11-19T01:45:34.194ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10822024-11-19T01:45:34.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10832024-11-19T01:45:34.196ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10842024-11-19T01:45:34.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
10852024-11-19T01:45:34.203ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 672.964914ms iteration = 27 status = {}
10862024-11-19T01:45:34.945ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8307576d-b386-4774-a07f-8acd76c1f2bc roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10872024-11-19T01:45:34.945ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8307576d-b386-4774-a07f-8acd76c1f2bc resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10882024-11-19T01:45:34.947ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2373 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8307576d-b386-4774-a07f-8acd76c1f2bc response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10892024-11-19T01:45:35.967ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8ab59708-0d97-4cae-ae34-41352e8ba97c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10902024-11-19T01:45:35.968ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8ab59708-0d97-4cae-ae34-41352e8ba97c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10912024-11-19T01:45:35.970ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2265 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8ab59708-0d97-4cae-ae34-41352e8ba97c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10922024-11-19T01:45:36.082ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = f3bad47f-0ac9-40dc-8ea0-e3f269ceed62 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
10932024-11-19T01:45:36.082ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = f3bad47f-0ac9-40dc-8ea0-e3f269ceed62 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
10942024-11-19T01:45:36.085ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3327 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = f3bad47f-0ac9-40dc-8ea0-e3f269ceed62 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
10952024-11-19T01:45:36.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 13bd3947-ad37-41c6-9ad7-b73b2a242c0c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10962024-11-19T01:45:36.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 13bd3947-ad37-41c6-9ad7-b73b2a242c0c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10972024-11-19T01:45:36.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1972 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 13bd3947-ad37-41c6-9ad7-b73b2a242c0c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10982024-11-19T01:45:38.015ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a8cd0dc2-f9a9-4ded-ab65-113606c28b76 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
10992024-11-19T01:45:38.016ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a8cd0dc2-f9a9-4ded-ab65-113606c28b76 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11002024-11-19T01:45:38.018ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2572 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a8cd0dc2-f9a9-4ded-ab65-113606c28b76 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11012024-11-19T01:45:39.039ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a7beed4f-2b18-42a6-83aa-a4adb14b508e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11022024-11-19T01:45:39.039ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a7beed4f-2b18-42a6-83aa-a4adb14b508e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11032024-11-19T01:45:39.041ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a7beed4f-2b18-42a6-83aa-a4adb14b508e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11042024-11-19T01:45:40.063ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bec22209-d785-4403-85cd-c8ff76bd6cb7 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11052024-11-19T01:45:40.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bec22209-d785-4403-85cd-c8ff76bd6cb7 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11062024-11-19T01:45:40.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2504 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bec22209-d785-4403-85cd-c8ff76bd6cb7 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11072024-11-19T01:45:41.088ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c7051559-597b-4802-8630-ba4d1532cce0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11082024-11-19T01:45:41.088ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c7051559-597b-4802-8630-ba4d1532cce0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11092024-11-19T01:45:41.090ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2393 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c7051559-597b-4802-8630-ba4d1532cce0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11102024-11-19T01:45:41.337ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:39137
11112024-11-19T01:45:41.337ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:39137 req_id = 5bde7700-82af-4bc5-9326-312591bc8601 roles = RoleSet { roles: {} } uri = /metrics/producers
11122024-11-19T01:45:41.337ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:39137 req_id = 5bde7700-82af-4bc5-9326-312591bc8601 resource = Database result = Ok(()) uri = /metrics/producers
11132024-11-19T01:45:41.386ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 302245ee-8785-4baf-86ed-0823410907ec
11142024-11-19T01:45:41.386ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48931 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:39137 req_id = 5bde7700-82af-4bc5-9326-312591bc8601 response_code = 201 uri = /metrics/producers
11152024-11-19T01:45:41.810ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1556 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 84ee9b7f-2484-4118-bc0f-b2a44f8cde8f response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
11162024-11-19T01:45:42.112ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37236259-a8d8-4296-bea3-be4712be3fa1 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11172024-11-19T01:45:42.112ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37236259-a8d8-4296-bea3-be4712be3fa1 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11182024-11-19T01:45:42.114ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2345 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37236259-a8d8-4296-bea3-be4712be3fa1 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11192024-11-19T01:45:43.138ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 98034e36-51b0-4852-9440-6020655110d0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11202024-11-19T01:45:43.138ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 98034e36-51b0-4852-9440-6020655110d0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11212024-11-19T01:45:43.141ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2819 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 98034e36-51b0-4852-9440-6020655110d0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11222024-11-19T01:45:43.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
11232024-11-19T01:45:43.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11242024-11-19T01:45:43.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11252024-11-19T01:45:43.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
11262024-11-19T01:45:43.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11272024-11-19T01:45:43.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11282024-11-19T01:45:43.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11292024-11-19T01:45:43.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11302024-11-19T01:45:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11312024-11-19T01:45:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11322024-11-19T01:45:43.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11332024-11-19T01:45:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11342024-11-19T01:45:43.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11352024-11-19T01:45:43.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11362024-11-19T01:45:43.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11372024-11-19T01:45:43.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
11382024-11-19T01:45:43.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11392024-11-19T01:45:43.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11402024-11-19T01:45:43.551ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
11412024-11-19T01:45:43.551ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.463893ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
11422024-11-19T01:45:43.563ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.258579ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11432024-11-19T01:45:44.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 81b8a24a-c466-4bc1-b603-9f30e942a358 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11442024-11-19T01:45:44.164ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 81b8a24a-c466-4bc1-b603-9f30e942a358 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11452024-11-19T01:45:44.166ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2554 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 81b8a24a-c466-4bc1-b603-9f30e942a358 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11462024-11-19T01:45:45.186ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c9e5867-88dd-49b0-8838-57426c759c79 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11472024-11-19T01:45:45.186ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c9e5867-88dd-49b0-8838-57426c759c79 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11482024-11-19T01:45:45.188ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2243 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c9e5867-88dd-49b0-8838-57426c759c79 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11492024-11-19T01:45:46.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c60c92d3-62ae-4111-b858-7e1856ccbf93 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11502024-11-19T01:45:46.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c60c92d3-62ae-4111-b858-7e1856ccbf93 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11512024-11-19T01:45:46.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1971 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c60c92d3-62ae-4111-b858-7e1856ccbf93 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11522024-11-19T01:45:46.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = f5274419-c3f5-4c04-90bd-98f1e62a581f roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
11532024-11-19T01:45:46.802ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = f5274419-c3f5-4c04-90bd-98f1e62a581f resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
11542024-11-19T01:45:46.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3071 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = f5274419-c3f5-4c04-90bd-98f1e62a581f response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
11552024-11-19T01:45:46.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 95a4ad2a-7a13-4706-9414-727d738edca0 roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
11562024-11-19T01:45:46.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 95a4ad2a-7a13-4706-9414-727d738edca0 resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
11572024-11-19T01:45:46.807ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2425 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 95a4ad2a-7a13-4706-9414-727d738edca0 response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
11582024-11-19T01:45:47.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6b5ea584-fc6d-4832-a1d4-03fda8d5fa93 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11592024-11-19T01:45:47.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6b5ea584-fc6d-4832-a1d4-03fda8d5fa93 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11602024-11-19T01:45:47.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2383 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6b5ea584-fc6d-4832-a1d4-03fda8d5fa93 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11612024-11-19T01:45:47.749ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:41958
11622024-11-19T01:45:47.749ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41958 req_id = bf025c17-17e8-4409-bdd8-236bb2a43dbc roles = RoleSet { roles: {} } uri = /metrics/producers
11632024-11-19T01:45:47.750ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41958 req_id = bf025c17-17e8-4409-bdd8-236bb2a43dbc resource = Database result = Ok(()) uri = /metrics/producers
11642024-11-19T01:45:47.764ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 128022c3-c3e6-460c-994e-e9c9cf944010
11652024-11-19T01:45:47.764ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 14324 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41958 req_id = bf025c17-17e8-4409-bdd8-236bb2a43dbc response_code = 201 uri = /metrics/producers
11662024-11-19T01:45:48.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7b6f559f-5006-441f-b78f-a4670f0c6fe4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11672024-11-19T01:45:48.258ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7b6f559f-5006-441f-b78f-a4670f0c6fe4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11682024-11-19T01:45:48.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2346 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7b6f559f-5006-441f-b78f-a4670f0c6fe4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11692024-11-19T01:45:49.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 66c55333-65b0-4382-b147-d2d65cacd6fd roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11702024-11-19T01:45:49.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 66c55333-65b0-4382-b147-d2d65cacd6fd resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11712024-11-19T01:45:49.285ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2710 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 66c55333-65b0-4382-b147-d2d65cacd6fd response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11722024-11-19T01:45:50.306ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9be79610-bb40-4256-a4a7-120de6e6f918 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11732024-11-19T01:45:50.307ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9be79610-bb40-4256-a4a7-120de6e6f918 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11742024-11-19T01:45:50.309ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2161 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9be79610-bb40-4256-a4a7-120de6e6f918 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11752024-11-19T01:45:51.332ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6c3c7e01-c36e-4149-971e-b06456e0dc06 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11762024-11-19T01:45:51.332ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6c3c7e01-c36e-4149-971e-b06456e0dc06 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11772024-11-19T01:45:51.334ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2583 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6c3c7e01-c36e-4149-971e-b06456e0dc06 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11782024-11-19T01:45:52.357ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 918cce6f-25a5-4a7a-88b6-ce4a69f4484b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11792024-11-19T01:45:52.357ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 918cce6f-25a5-4a7a-88b6-ce4a69f4484b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11802024-11-19T01:45:52.358ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1979 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 918cce6f-25a5-4a7a-88b6-ce4a69f4484b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11812024-11-19T01:45:53.380ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f46ef7cc-b8d5-4737-974e-f31e2674b69e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11822024-11-19T01:45:53.380ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f46ef7cc-b8d5-4737-974e-f31e2674b69e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11832024-11-19T01:45:53.382ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2428 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f46ef7cc-b8d5-4737-974e-f31e2674b69e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
11842024-11-19T01:45:53.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
11852024-11-19T01:45:53.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11862024-11-19T01:45:53.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11872024-11-19T01:45:53.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11882024-11-19T01:45:53.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
11892024-11-19T01:45:53.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11902024-11-19T01:45:53.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11912024-11-19T01:45:53.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11922024-11-19T01:45:53.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11932024-11-19T01:45:53.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11942024-11-19T01:45:53.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11952024-11-19T01:45:53.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11962024-11-19T01:45:53.535ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.403521ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11972024-11-19T01:45:53.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11982024-11-19T01:45:53.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11992024-11-19T01:45:53.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12002024-11-19T01:45:53.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
12012024-11-19T01:45:53.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12022024-11-19T01:45:53.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12032024-11-19T01:45:53.548ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
12042024-11-19T01:45:53.548ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 18.537042ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
12052024-11-19T01:45:54.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f192f02d-766c-4090-8614-2b34b43d0cf1 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12062024-11-19T01:45:54.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f192f02d-766c-4090-8614-2b34b43d0cf1 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12072024-11-19T01:45:54.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1807 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f192f02d-766c-4090-8614-2b34b43d0cf1 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12082024-11-19T01:45:55.427ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c14b72f6-3c12-49f8-bfe4-9ff5d1f56c59 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12092024-11-19T01:45:55.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c14b72f6-3c12-49f8-bfe4-9ff5d1f56c59 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12102024-11-19T01:45:55.429ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2375 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c14b72f6-3c12-49f8-bfe4-9ff5d1f56c59 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12112024-11-19T01:45:56.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2577e5d9-dd8c-4648-9528-aae88ea3075f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12122024-11-19T01:45:56.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2577e5d9-dd8c-4648-9528-aae88ea3075f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12132024-11-19T01:45:56.454ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2059 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2577e5d9-dd8c-4648-9528-aae88ea3075f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12142024-11-19T01:45:56.810ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1462 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 18d63ade-8747-4fc6-abd0-997ac9b54317 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
12152024-11-19T01:45:57.475ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c708da3a-f4ee-4bcd-824a-90176803fe48 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12162024-11-19T01:45:57.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c708da3a-f4ee-4bcd-824a-90176803fe48 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12172024-11-19T01:45:57.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2745 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c708da3a-f4ee-4bcd-824a-90176803fe48 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12182024-11-19T01:45:58.499ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 08aad8bb-651b-4751-b784-9d0049577ad6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12192024-11-19T01:45:58.500ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 08aad8bb-651b-4751-b784-9d0049577ad6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12202024-11-19T01:45:58.502ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2494 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 08aad8bb-651b-4751-b784-9d0049577ad6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12212024-11-19T01:45:58.559ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:49446
12222024-11-19T01:45:58.560ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49446 req_id = fb21fe23-6966-4db3-b1dc-a0cc0acffc93 roles = RoleSet { roles: {} } uri = /metrics/producers
12232024-11-19T01:45:58.560ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49446 req_id = fb21fe23-6966-4db3-b1dc-a0cc0acffc93 resource = Database result = Ok(()) uri = /metrics/producers
12242024-11-19T01:45:58.603ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 60df7fc2-269e-4e73-a446-d55cd69c6f17
12252024-11-19T01:45:58.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43993 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49446 req_id = fb21fe23-6966-4db3-b1dc-a0cc0acffc93 response_code = 201 uri = /metrics/producers
12262024-11-19T01:45:59.523ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 450f0192-3384-46d1-baa5-f8a52de3d1d5 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12272024-11-19T01:45:59.523ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 450f0192-3384-46d1-baa5-f8a52de3d1d5 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12282024-11-19T01:45:59.525ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2141 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 450f0192-3384-46d1-baa5-f8a52de3d1d5 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12292024-11-19T01:46:00.547ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 833d6811-9bae-43b7-a932-82c7d8312b6c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12302024-11-19T01:46:00.548ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 833d6811-9bae-43b7-a932-82c7d8312b6c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12312024-11-19T01:46:00.549ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2297 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 833d6811-9bae-43b7-a932-82c7d8312b6c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12322024-11-19T01:46:01.574ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8b5bcdf3-a17c-4763-8716-ae96f4520e0a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12332024-11-19T01:46:01.574ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8b5bcdf3-a17c-4763-8716-ae96f4520e0a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12342024-11-19T01:46:01.576ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2437 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8b5bcdf3-a17c-4763-8716-ae96f4520e0a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12352024-11-19T01:46:01.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 240ad66e-01f7-4956-a395-99bf04e8049d roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
12362024-11-19T01:46:01.802ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 240ad66e-01f7-4956-a395-99bf04e8049d resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
12372024-11-19T01:46:01.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3048 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 240ad66e-01f7-4956-a395-99bf04e8049d response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
12382024-11-19T01:46:01.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 01a5715a-77fa-4da4-8e9f-be49e6f4dfc4 roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
12392024-11-19T01:46:01.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 01a5715a-77fa-4da4-8e9f-be49e6f4dfc4 resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
12402024-11-19T01:46:01.807ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2164 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 01a5715a-77fa-4da4-8e9f-be49e6f4dfc4 response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
12412024-11-19T01:46:02.597ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9cd4c1ae-717b-4508-bf83-824af4a1ea71 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12422024-11-19T01:46:02.597ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9cd4c1ae-717b-4508-bf83-824af4a1ea71 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12432024-11-19T01:46:02.599ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2546 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9cd4c1ae-717b-4508-bf83-824af4a1ea71 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
12442024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 28 reason = Timeout
12452024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
12462024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
12472024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
12482024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12492024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
12502024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
12512024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
12522024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
12532024-11-19T01:46:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
12542024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
12552024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12562024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
12572024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
12582024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
12592024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 29 reason = Timeout
12602024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12612024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12622024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12632024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12642024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12652024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12662024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
12672024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12682024-11-19T01:46:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12692024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
12702024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
12712024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
12722024-11-19T01:46:03.532ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
12732024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
12742024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12752024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12762024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12772024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
12782024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12792024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12802024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12812024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12822024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12832024-11-19T01:46:03.532ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 699370212 } })
12842024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12852024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
12862024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12872024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
12882024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12892024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
12902024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
12912024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
12922024-11-19T01:46:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
12932024-11-19T01:46:03.532ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
12942024-11-19T01:46:03.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
12952024-11-19T01:46:03.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12962024-11-19T01:46:03.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
12972024-11-19T01:46:03.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12982024-11-19T01:46:03.534ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.92437ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
12992024-11-19T01:46:03.534ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 701415948 } })
13002024-11-19T01:46:03.534ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
13012024-11-19T01:46:03.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13022024-11-19T01:46:03.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13032024-11-19T01:46:03.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13042024-11-19T01:46:03.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13052024-11-19T01:46:03.537ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13062024-11-19T01:46:03.538ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13072024-11-19T01:46:03.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13082024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13092024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13102024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13112024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13122024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
13132024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13142024-11-19T01:46:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13152024-11-19T01:46:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
13162024-11-19T01:46:03.540ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
13172024-11-19T01:46:03.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13182024-11-19T01:46:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
13192024-11-19T01:46:03.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13202024-11-19T01:46:03.542ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13212024-11-19T01:46:03.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13222024-11-19T01:46:03.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13232024-11-19T01:46:03.544ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 12.376862ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13242024-11-19T01:46:03.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13252024-11-19T01:46:03.545ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13262024-11-19T01:46:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13272024-11-19T01:46:03.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13282024-11-19T01:46:03.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13292024-11-19T01:46:03.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13302024-11-19T01:46:03.546ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1813, tv_nsec: 713680838 } })
13312024-11-19T01:46:03.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13322024-11-19T01:46:03.547ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
13332024-11-19T01:46:03.548ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13342024-11-19T01:46:03.548ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13352024-11-19T01:46:03.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13362024-11-19T01:46:03.553ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13372024-11-19T01:46:03.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
13382024-11-19T01:46:03.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13392024-11-19T01:46:03.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13402024-11-19T01:46:03.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
13412024-11-19T01:46:03.555ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
13422024-11-19T01:46:03.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
13432024-11-19T01:46:03.565ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
13442024-11-19T01:46:03.565ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 35.79455ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
13452024-11-19T01:46:03.573ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13462024-11-19T01:46:03.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 42.077692ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
13472024-11-19T01:46:03.575ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13482024-11-19T01:46:03.575ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
13492024-11-19T01:46:03.575ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13502024-11-19T01:46:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13512024-11-19T01:46:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
13522024-11-19T01:46:03.576ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13532024-11-19T01:46:03.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
13542024-11-19T01:46:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13552024-11-19T01:46:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
13562024-11-19T01:46:03.581ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13572024-11-19T01:46:03.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13582024-11-19T01:46:03.582ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 50.063026ms iteration = 28 status = {"errors":[],"records_set_to_done":[]}
13592024-11-19T01:46:03.583ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
13602024-11-19T01:46:03.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
13612024-11-19T01:46:03.593ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13622024-11-19T01:46:03.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13632024-11-19T01:46:03.596ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
13642024-11-19T01:46:03.596ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 64.695087ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
13652024-11-19T01:46:03.618ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
13662024-11-19T01:46:03.618ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13672024-11-19T01:46:03.618ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a180b8f-f624-47d4-9bb3-5dc8e29cb255", "content-length": "26", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
13682024-11-19T01:46:03.619ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13692024-11-19T01:46:03.621ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13702024-11-19T01:46:03.622ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bdd5f039-ab8e-4272-8c83-41e466d72515 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
13712024-11-19T01:46:03.622ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bdd5f039-ab8e-4272-8c83-41e466d72515 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
13722024-11-19T01:46:03.631ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
13732024-11-19T01:46:03.631ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13742024-11-19T01:46:03.631ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1f6165c-00aa-4f74-a496-e61033b6a823", "content-length": "26", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
13752024-11-19T01:46:03.631ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13762024-11-19T01:46:03.633ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13772024-11-19T01:46:03.658ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13782024-11-19T01:46:03.658ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13792024-11-19T01:46:03.659ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1266e28-e62d-4156-b307-9023acd017e6", "content-length": "372", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
13802024-11-19T01:46:03.659ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13812024-11-19T01:46:03.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13822024-11-19T01:46:03.662ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13832024-11-19T01:46:03.662ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
13842024-11-19T01:46:03.662ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13852024-11-19T01:46:03.662ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
13862024-11-19T01:46:03.662ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
13872024-11-19T01:46:03.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
13882024-11-19T01:46:03.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
13892024-11-19T01:46:03.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
13902024-11-19T01:46:03.664ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13912024-11-19T01:46:03.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43525 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bdd5f039-ab8e-4272-8c83-41e466d72515 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
13922024-11-19T01:46:03.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13932024-11-19T01:46:03.666ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13942024-11-19T01:46:03.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
13952024-11-19T01:46:03.677ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
13962024-11-19T01:46:03.677ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13972024-11-19T01:46:03.677ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7872814-4748-4766-9fe8-fb81578abaa9", "content-length": "26", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
13982024-11-19T01:46:03.677ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13992024-11-19T01:46:03.680ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
14002024-11-19T01:46:03.690ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14012024-11-19T01:46:03.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14022024-11-19T01:46:03.692ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14032024-11-19T01:46:03.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14042024-11-19T01:46:03.698ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 165.201704ms iteration = 28 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
14052024-11-19T01:46:03.698ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 165.206245ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
14062024-11-19T01:46:03.698ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14072024-11-19T01:46:03.698ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 167.742ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
14082024-11-19T01:46:03.700ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
14092024-11-19T01:46:03.700ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
14102024-11-19T01:46:03.700ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 167.894183ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
14112024-11-19T01:46:03.700ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
14122024-11-19T01:46:03.700ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14132024-11-19T01:46:03.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
14142024-11-19T01:46:03.711ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
14152024-11-19T01:46:03.711ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
14162024-11-19T01:46:03.712ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9ddd1709-ba67-4c0f-8bb5-337367185549", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14172024-11-19T01:46:03.712ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14182024-11-19T01:46:03.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14192024-11-19T01:46:03.714ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14202024-11-19T01:46:03.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14212024-11-19T01:46:03.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14222024-11-19T01:46:03.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14232024-11-19T01:46:03.722ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 192.557263ms iteration = 28 status = {}
14242024-11-19T01:46:03.724ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
14252024-11-19T01:46:03.724ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14262024-11-19T01:46:03.725ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "abc701cd-f7a2-4ae8-aed7-e4e82b6b903c", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14272024-11-19T01:46:03.726ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 194.379496ms iteration = 30 status = {}
14282024-11-19T01:46:03.728ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14292024-11-19T01:46:03.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
14302024-11-19T01:46:03.730ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14312024-11-19T01:46:03.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14322024-11-19T01:46:03.736ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
14332024-11-19T01:46:03.737ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff21c1b7-2d7d-4176-be5d-07e93821c6d3", "content-length": "2", "date": "Tue, 19 Nov 2024 01:46:02 GMT"} })
14342024-11-19T01:46:03.738ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
14352024-11-19T01:46:03.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
14362024-11-19T01:46:03.762ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 231.271326ms iteration = 32 status = {"physical_disks_added":0}
14372024-11-19T01:46:03.825ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
14382024-11-19T01:46:03.827ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 296.085405ms iteration = 28 status = 0
14392024-11-19T01:46:03.827ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 297.851556ms iteration = 28 status = null
14402024-11-19T01:46:03.878ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "80af79f1-57a9-43f3-934e-d789f9a26a97", "content-length": "691", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14412024-11-19T01:46:03.879ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
14422024-11-19T01:46:03.879ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
14432024-11-19T01:46:03.879ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
14442024-11-19T01:46:03.881ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 349.879086ms iteration = 29 status = {}
14452024-11-19T01:46:03.884ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14462024-11-19T01:46:03.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14472024-11-19T01:46:03.886ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
14482024-11-19T01:46:03.886ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14492024-11-19T01:46:03.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14502024-11-19T01:46:03.896ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
14512024-11-19T01:46:03.897ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e3701e6-4cd3-4089-9807-a0801ba1c460", "content-length": "134", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14522024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
14532024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
14542024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
14552024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
14562024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14572024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14582024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14592024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14602024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14612024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14622024-11-19T01:46:03.897ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
14632024-11-19T01:46:03.897ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14642024-11-19T01:46:03.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14652024-11-19T01:46:03.900ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
14662024-11-19T01:46:03.900ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "37694011-ea35-4439-a69f-3b830b581c0b", "content-length": "2", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14672024-11-19T01:46:03.900ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
14682024-11-19T01:46:03.900ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "78f99afb-4470-480c-b979-69efa78a0d32", "content-length": "51", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14692024-11-19T01:46:03.900ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
14702024-11-19T01:46:03.900ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14712024-11-19T01:46:04.000ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "234a91e8-b42b-4984-8315-027f56398431", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14722024-11-19T01:46:04.000ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
14732024-11-19T01:46:04.000ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14742024-11-19T01:46:04.000ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f90f10ac-82ea-445c-8f37-c5a943e2d89c", "content-length": "682", "date": "Tue, 19 Nov 2024 01:46:03 GMT"} })
14752024-11-19T01:46:04.000ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14762024-11-19T01:46:04.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14772024-11-19T01:46:04.003ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14782024-11-19T01:46:04.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14792024-11-19T01:46:04.007ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14802024-11-19T01:46:04.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14812024-11-19T01:46:04.009ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14822024-11-19T01:46:04.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14832024-11-19T01:46:04.017ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 485.774426ms iteration = 28 status = {}
14842024-11-19T01:46:04.690ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 433c40d0-2948-4a7a-b04c-eb23c9c5c21d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14852024-11-19T01:46:04.691ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 433c40d0-2948-4a7a-b04c-eb23c9c5c21d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14862024-11-19T01:46:04.693ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2352 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 433c40d0-2948-4a7a-b04c-eb23c9c5c21d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14872024-11-19T01:46:05.811ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8d4bf798-2ccb-4eb6-987f-624c2d752a78 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14882024-11-19T01:46:05.812ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8d4bf798-2ccb-4eb6-987f-624c2d752a78 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14892024-11-19T01:46:05.814ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2218 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8d4bf798-2ccb-4eb6-987f-624c2d752a78 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14902024-11-19T01:46:06.117ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = c76c3763-2a1d-4758-9738-7dac6902e73b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
14912024-11-19T01:46:06.117ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = c76c3763-2a1d-4758-9738-7dac6902e73b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
14922024-11-19T01:46:06.120ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3023 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = c76c3763-2a1d-4758-9738-7dac6902e73b response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
14932024-11-19T01:46:06.837ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 745ce403-8f81-4146-99d8-12d5fe396603 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14942024-11-19T01:46:06.838ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 745ce403-8f81-4146-99d8-12d5fe396603 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14952024-11-19T01:46:06.840ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2615 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 745ce403-8f81-4146-99d8-12d5fe396603 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14962024-11-19T01:46:07.861ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1c761747-ca2d-4463-8380-24f006cd5b73 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14972024-11-19T01:46:07.862ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1c761747-ca2d-4463-8380-24f006cd5b73 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14982024-11-19T01:46:07.864ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2929 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1c761747-ca2d-4463-8380-24f006cd5b73 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
14992024-11-19T01:46:08.887ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ab6b44cb-191a-4481-850d-3a002692feb9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15002024-11-19T01:46:08.887ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ab6b44cb-191a-4481-850d-3a002692feb9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15012024-11-19T01:46:08.889ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2435 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ab6b44cb-191a-4481-850d-3a002692feb9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15022024-11-19T01:46:09.085ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:41153
15032024-11-19T01:46:09.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41153 req_id = 3771afc3-3c69-44a0-aecf-9c937258583c roles = RoleSet { roles: {} } uri = /metrics/producers
15042024-11-19T01:46:09.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41153 req_id = 3771afc3-3c69-44a0-aecf-9c937258583c resource = Database result = Ok(()) uri = /metrics/producers
15052024-11-19T01:46:09.129ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = e775bb0f-d2b5-4c13-b4f1-f5a61d1d3098
15062024-11-19T01:46:09.129ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44459 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:41153 req_id = 3771afc3-3c69-44a0-aecf-9c937258583c response_code = 201 uri = /metrics/producers
15072024-11-19T01:46:09.909ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0fdb466f-0201-4e39-9438-b799c32f0290 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15082024-11-19T01:46:09.910ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0fdb466f-0201-4e39-9438-b799c32f0290 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15092024-11-19T01:46:09.912ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2783 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0fdb466f-0201-4e39-9438-b799c32f0290 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15102024-11-19T01:46:10.934ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd3b343e-8586-4809-ba62-0c549500bc31 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15112024-11-19T01:46:10.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd3b343e-8586-4809-ba62-0c549500bc31 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15122024-11-19T01:46:10.936ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2228 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd3b343e-8586-4809-ba62-0c549500bc31 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15132024-11-19T01:46:11.809ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1549 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = ed971e5a-a7b2-4808-a684-f5d742f6467f response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
15142024-11-19T01:46:11.958ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 495add65-2d8d-4751-bf5f-8169a400445d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15152024-11-19T01:46:11.958ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 495add65-2d8d-4751-bf5f-8169a400445d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15162024-11-19T01:46:11.960ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2393 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 495add65-2d8d-4751-bf5f-8169a400445d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15172024-11-19T01:46:12.983ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6a11df6f-9f47-4de4-a8a4-70891d625f63 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15182024-11-19T01:46:12.983ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6a11df6f-9f47-4de4-a8a4-70891d625f63 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15192024-11-19T01:46:12.985ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2544 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6a11df6f-9f47-4de4-a8a4-70891d625f63 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15202024-11-19T01:46:13.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
15212024-11-19T01:46:13.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15222024-11-19T01:46:13.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15232024-11-19T01:46:13.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
15242024-11-19T01:46:13.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15252024-11-19T01:46:13.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15262024-11-19T01:46:13.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15272024-11-19T01:46:13.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15282024-11-19T01:46:13.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15292024-11-19T01:46:13.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15302024-11-19T01:46:13.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15312024-11-19T01:46:13.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15322024-11-19T01:46:13.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15332024-11-19T01:46:13.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15342024-11-19T01:46:13.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15352024-11-19T01:46:13.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
15362024-11-19T01:46:13.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15372024-11-19T01:46:13.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15382024-11-19T01:46:13.582ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
15392024-11-19T01:46:13.582ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 51.539612ms iteration = 83 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
15402024-11-19T01:46:13.595ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 62.921335ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15412024-11-19T01:46:14.006ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = af7e291b-7e4e-43c0-a1e5-12c40954a1a8 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15422024-11-19T01:46:14.007ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = af7e291b-7e4e-43c0-a1e5-12c40954a1a8 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15432024-11-19T01:46:14.009ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2206 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = af7e291b-7e4e-43c0-a1e5-12c40954a1a8 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15442024-11-19T01:46:15.030ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1037c951-a749-42d2-9e9e-cfbdc00666d2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15452024-11-19T01:46:15.030ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1037c951-a749-42d2-9e9e-cfbdc00666d2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15462024-11-19T01:46:15.032ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2390 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1037c951-a749-42d2-9e9e-cfbdc00666d2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15472024-11-19T01:46:16.056ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 69f55356-7283-4c6e-881d-1eb9ae99ef4c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15482024-11-19T01:46:16.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 69f55356-7283-4c6e-881d-1eb9ae99ef4c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15492024-11-19T01:46:16.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1980 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 69f55356-7283-4c6e-881d-1eb9ae99ef4c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15502024-11-19T01:46:16.802ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 3579ca9c-f286-4f14-b03f-6a0f91ffe1f5 roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
15512024-11-19T01:46:16.803ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 3579ca9c-f286-4f14-b03f-6a0f91ffe1f5 resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
15522024-11-19T01:46:16.805ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3002 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 3579ca9c-f286-4f14-b03f-6a0f91ffe1f5 response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
15532024-11-19T01:46:16.805ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = aba1cedc-f7cf-4d1e-9287-b8dec1220357 roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
15542024-11-19T01:46:16.806ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = aba1cedc-f7cf-4d1e-9287-b8dec1220357 resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
15552024-11-19T01:46:16.808ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2219 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = aba1cedc-f7cf-4d1e-9287-b8dec1220357 response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
15562024-11-19T01:46:17.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f86d7183-82b9-4506-b60f-e459ff529790 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15572024-11-19T01:46:17.081ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f86d7183-82b9-4506-b60f-e459ff529790 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15582024-11-19T01:46:17.083ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2311 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f86d7183-82b9-4506-b60f-e459ff529790 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15592024-11-19T01:46:18.105ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7095587c-3288-4e9a-b047-dbae65008eb8 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15602024-11-19T01:46:18.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7095587c-3288-4e9a-b047-dbae65008eb8 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15612024-11-19T01:46:18.107ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2294 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7095587c-3288-4e9a-b047-dbae65008eb8 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15622024-11-19T01:46:19.129ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0efe9861-76f6-46d7-a31d-fec632e261bb roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15632024-11-19T01:46:19.130ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0efe9861-76f6-46d7-a31d-fec632e261bb resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15642024-11-19T01:46:19.132ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2487 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0efe9861-76f6-46d7-a31d-fec632e261bb response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15652024-11-19T01:46:20.153ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9cb5f0e3-2d70-43e9-b860-ef7af64c103a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15662024-11-19T01:46:20.153ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9cb5f0e3-2d70-43e9-b860-ef7af64c103a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15672024-11-19T01:46:20.156ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2632 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9cb5f0e3-2d70-43e9-b860-ef7af64c103a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15682024-11-19T01:46:21.177ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1dab4b2e-da87-4fd7-a46f-120779caea10 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15692024-11-19T01:46:21.177ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1dab4b2e-da87-4fd7-a46f-120779caea10 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15702024-11-19T01:46:21.179ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2093 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1dab4b2e-da87-4fd7-a46f-120779caea10 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15712024-11-19T01:46:22.201ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 75448d60-b38c-4c50-9862-52315c93d2df roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15722024-11-19T01:46:22.201ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 75448d60-b38c-4c50-9862-52315c93d2df resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15732024-11-19T01:46:22.203ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2267 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 75448d60-b38c-4c50-9862-52315c93d2df response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15742024-11-19T01:46:23.225ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 60d00217-91d3-4b6a-bb77-453a7451a059 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15752024-11-19T01:46:23.225ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 60d00217-91d3-4b6a-bb77-453a7451a059 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15762024-11-19T01:46:23.227ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2413 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 60d00217-91d3-4b6a-bb77-453a7451a059 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15772024-11-19T01:46:23.531ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
15782024-11-19T01:46:23.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15792024-11-19T01:46:23.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15802024-11-19T01:46:23.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
15812024-11-19T01:46:23.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15822024-11-19T01:46:23.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15832024-11-19T01:46:23.534ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15842024-11-19T01:46:23.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15852024-11-19T01:46:23.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15862024-11-19T01:46:23.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15872024-11-19T01:46:23.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15882024-11-19T01:46:23.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15892024-11-19T01:46:23.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15902024-11-19T01:46:23.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15912024-11-19T01:46:23.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.294279ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
15922024-11-19T01:46:23.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15932024-11-19T01:46:23.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
15942024-11-19T01:46:23.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15952024-11-19T01:46:23.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15962024-11-19T01:46:23.584ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
15972024-11-19T01:46:23.584ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.203531ms iteration = 84 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
15982024-11-19T01:46:24.248ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = cdc2cb3f-1168-4626-8ba8-b4b640b63edf roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
15992024-11-19T01:46:24.248ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = cdc2cb3f-1168-4626-8ba8-b4b640b63edf resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16002024-11-19T01:46:24.250ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = cdc2cb3f-1168-4626-8ba8-b4b640b63edf response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16012024-11-19T01:46:25.272ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0bccaaf6-06dc-4276-acd3-4ddc11268ec9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16022024-11-19T01:46:25.272ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0bccaaf6-06dc-4276-acd3-4ddc11268ec9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16032024-11-19T01:46:25.274ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2388 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0bccaaf6-06dc-4276-acd3-4ddc11268ec9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16042024-11-19T01:46:26.297ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 363b0093-9b25-4dfa-be0a-9667c141da08 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16052024-11-19T01:46:26.297ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 363b0093-9b25-4dfa-be0a-9667c141da08 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16062024-11-19T01:46:26.299ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2071 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 363b0093-9b25-4dfa-be0a-9667c141da08 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16072024-11-19T01:46:26.811ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1795 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 4b7139fe-158c-4fcd-b385-343f924c4534 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
16082024-11-19T01:46:27.322ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a9c5ca63-202b-45bc-a4d7-2159efbd11ce roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16092024-11-19T01:46:27.322ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a9c5ca63-202b-45bc-a4d7-2159efbd11ce resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16102024-11-19T01:46:27.324ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2215 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a9c5ca63-202b-45bc-a4d7-2159efbd11ce response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16112024-11-19T01:46:28.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d8933bd-3d87-437a-8fdc-f70f7509ba24 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16122024-11-19T01:46:28.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d8933bd-3d87-437a-8fdc-f70f7509ba24 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16132024-11-19T01:46:28.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2440 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d8933bd-3d87-437a-8fdc-f70f7509ba24 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16142024-11-19T01:46:29.370ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e23f9198-273d-47dc-9a0a-53c579104df0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16152024-11-19T01:46:29.371ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e23f9198-273d-47dc-9a0a-53c579104df0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16162024-11-19T01:46:29.373ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e23f9198-273d-47dc-9a0a-53c579104df0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16172024-11-19T01:46:30.394ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 88e77e09-51d8-417a-b5f4-3603641f5250 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16182024-11-19T01:46:30.395ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 88e77e09-51d8-417a-b5f4-3603641f5250 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16192024-11-19T01:46:30.396ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2139 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 88e77e09-51d8-417a-b5f4-3603641f5250 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16202024-11-19T01:46:31.417ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2d24f6c1-8b59-4ba2-b772-9ac6e007fb2a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16212024-11-19T01:46:31.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2d24f6c1-8b59-4ba2-b772-9ac6e007fb2a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16222024-11-19T01:46:31.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3003 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2d24f6c1-8b59-4ba2-b772-9ac6e007fb2a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16232024-11-19T01:46:32.441ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 415b2c6b-20b0-4933-9e33-77eecf0df8d0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16242024-11-19T01:46:32.441ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 415b2c6b-20b0-4933-9e33-77eecf0df8d0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16252024-11-19T01:46:32.443ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2638 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 415b2c6b-20b0-4933-9e33-77eecf0df8d0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16262024-11-19T01:46:33.465ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 682b8245-aaa7-452d-b20a-cfb1ca556d22 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16272024-11-19T01:46:33.466ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 682b8245-aaa7-452d-b20a-cfb1ca556d22 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16282024-11-19T01:46:33.471ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6090 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 682b8245-aaa7-452d-b20a-cfb1ca556d22 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
16292024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 15 reason = Timeout
16302024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 16 reason = Timeout
16312024-11-19T01:46:33.531ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-19 01:36:33.531079763 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
16322024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
16332024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 85 reason = Timeout
16342024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 17 reason = Timeout
16352024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 16 reason = Timeout
16362024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16372024-11-19T01:46:33.531ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120
16382024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 29 reason = Timeout
16392024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.22\u{b5}s iteration = 16 status = {"enabled":false,"target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120"}
16402024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 29 reason = Timeout
16412024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
16422024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 16 reason = Timeout
16432024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 16 reason = Timeout
16442024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 16 reason = Timeout
16452024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
16462024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
16472024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
16482024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
16492024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16502024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 17 reason = Timeout
16512024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 16 reason = Timeout
16522024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 29 reason = Timeout
16532024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
16542024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 29 reason = Timeout
16552024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16562024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
16572024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16582024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 15 reason = Timeout
16592024-11-19T01:46:33.531ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
16602024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
16612024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 30 reason = Timeout
16622024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16632024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 29 reason = Timeout
16642024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 29 reason = Timeout
16652024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16662024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16672024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
16682024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16692024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16702024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
16712024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
16722024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16732024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
16742024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
16752024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16762024-11-19T01:46:33.531ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("52e4d19a-b928-4138-8817-4fd8deeaf29e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 698201648 } })
16772024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
16782024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16792024-11-19T01:46:33.531ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e98d393-d940-4def-ba7c-d8c67ea23aa1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e48e23d9-6a0a-4bc6-9e0b-f83ba78952f1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 698257079 } })
16802024-11-19T01:46:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16812024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16822024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
16832024-11-19T01:46:33.532ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
16842024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 29 reason = Timeout
16852024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16862024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16872024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 85 reason = Timeout
16882024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 15 reason = Timeout
16892024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16902024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
16912024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 29 reason = Timeout
16922024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
16932024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16942024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16952024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
16962024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.610719ms iteration = 16 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
16972024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
16982024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.472556ms iteration = 16 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
16992024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
17002024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
17012024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17022024-11-19T01:46:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17032024-11-19T01:46:33.534ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 29 reason = Timeout
17042024-11-19T01:46:33.534ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
17052024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
17062024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 15 reason = Timeout
17072024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
17082024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
17092024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 29 reason = Timeout
17102024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17112024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17122024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 15 reason = Timeout
17132024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
17142024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
17152024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 29 reason = Timeout
17162024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
17172024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17182024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
17192024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 29 reason = Timeout
17202024-11-19T01:46:33.535ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
17212024-11-19T01:46:33.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
17222024-11-19T01:46:33.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17232024-11-19T01:46:33.536ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17242024-11-19T01:46:33.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17252024-11-19T01:46:33.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17262024-11-19T01:46:33.537ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-19 01:36:33.531079763 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
17272024-11-19T01:46:33.537ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.72354ms iteration = 15 status = {"expiration":"2024-11-19T01:36:33.531079763Z","pruned":[]}
17282024-11-19T01:46:33.538ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.996065ms iteration = 29 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
17292024-11-19T01:46:33.539ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 705334575 } })
17302024-11-19T01:46:33.539ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
17312024-11-19T01:46:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17322024-11-19T01:46:33.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17332024-11-19T01:46:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17342024-11-19T01:46:33.541ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17352024-11-19T01:46:33.545ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17362024-11-19T01:46:33.545ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17372024-11-19T01:46:33.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17382024-11-19T01:46:33.548ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 714608201 } })
17392024-11-19T01:46:33.548ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
17402024-11-19T01:46:33.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17412024-11-19T01:46:33.549ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17422024-11-19T01:46:33.550ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17432024-11-19T01:46:33.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
17442024-11-19T01:46:33.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17452024-11-19T01:46:33.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
17462024-11-19T01:46:33.553ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
17472024-11-19T01:46:33.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17482024-11-19T01:46:33.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
17492024-11-19T01:46:33.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
17502024-11-19T01:46:33.554ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17512024-11-19T01:46:33.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17522024-11-19T01:46:33.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17532024-11-19T01:46:33.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17542024-11-19T01:46:33.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17552024-11-19T01:46:33.557ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17562024-11-19T01:46:33.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17572024-11-19T01:46:33.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17582024-11-19T01:46:33.560ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 25.106699ms iteration = 15 status = {"errors":[],"found_port_ok":[]}
17592024-11-19T01:46:33.562ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17602024-11-19T01:46:33.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17612024-11-19T01:46:33.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
17622024-11-19T01:46:33.563ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17632024-11-19T01:46:33.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17642024-11-19T01:46:33.565ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1843, tv_nsec: 732040463 } })
17652024-11-19T01:46:33.566ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
17662024-11-19T01:46:33.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17672024-11-19T01:46:33.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17682024-11-19T01:46:33.572ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
17692024-11-19T01:46:33.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17702024-11-19T01:46:33.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17712024-11-19T01:46:33.576ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17722024-11-19T01:46:33.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17732024-11-19T01:46:33.578ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17742024-11-19T01:46:33.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
17752024-11-19T01:46:33.581ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17762024-11-19T01:46:33.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17772024-11-19T01:46:33.590ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17782024-11-19T01:46:33.592ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17792024-11-19T01:46:33.592ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17802024-11-19T01:46:33.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17812024-11-19T01:46:33.597ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
17822024-11-19T01:46:33.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
17832024-11-19T01:46:33.602ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 67.832553ms iteration = 29 status = {"errors":[],"records_set_to_done":[]}
17842024-11-19T01:46:33.602ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17852024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
17862024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
17872024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
17882024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 70.112924ms iteration = 29 status = {"errors":[],"garbage_collect_requested":[]}
17892024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
17902024-11-19T01:46:33.605ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
17912024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 70.863687ms iteration = 15 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
17922024-11-19T01:46:33.605ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
17932024-11-19T01:46:33.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
17942024-11-19T01:46:33.608ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17952024-11-19T01:46:33.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17962024-11-19T01:46:33.609ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17972024-11-19T01:46:33.609ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
17982024-11-19T01:46:33.609ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
17992024-11-19T01:46:33.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18002024-11-19T01:46:33.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18012024-11-19T01:46:33.614ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
18022024-11-19T01:46:33.614ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 81.6709ms iteration = 29 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
18032024-11-19T01:46:33.614ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
18042024-11-19T01:46:33.614ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
18052024-11-19T01:46:33.614ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
18062024-11-19T01:46:33.614ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18072024-11-19T01:46:33.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18082024-11-19T01:46:33.615ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18092024-11-19T01:46:33.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18102024-11-19T01:46:33.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18112024-11-19T01:46:33.617ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
18122024-11-19T01:46:33.617ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
18132024-11-19T01:46:33.617ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 86.414956ms iteration = 15 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
18142024-11-19T01:46:33.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18152024-11-19T01:46:33.619ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18162024-11-19T01:46:33.619ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18172024-11-19T01:46:33.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18182024-11-19T01:46:33.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18192024-11-19T01:46:33.621ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.632565ms iteration = 85 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18202024-11-19T01:46:33.621ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
18212024-11-19T01:46:33.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18222024-11-19T01:46:33.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
18232024-11-19T01:46:33.624ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18242024-11-19T01:46:33.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18252024-11-19T01:46:33.633ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18262024-11-19T01:46:33.637ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
18272024-11-19T01:46:33.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
18282024-11-19T01:46:33.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
18292024-11-19T01:46:33.638ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
18302024-11-19T01:46:33.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
18312024-11-19T01:46:33.640ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 109.03999ms iteration = 29 status = 0
18322024-11-19T01:46:33.649ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
18332024-11-19T01:46:33.649ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 118.152092ms iteration = 85 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
18342024-11-19T01:46:33.649ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
18352024-11-19T01:46:33.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
18362024-11-19T01:46:33.651ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 119.968535ms iteration = 29 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
18372024-11-19T01:46:33.651ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18382024-11-19T01:46:33.651ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
18392024-11-19T01:46:33.651ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
18402024-11-19T01:46:33.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
18412024-11-19T01:46:33.656ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 121.153147ms iteration = 29 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
18422024-11-19T01:46:33.656ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18432024-11-19T01:46:33.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18442024-11-19T01:46:33.661ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
18452024-11-19T01:46:33.661ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
18462024-11-19T01:46:33.661ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
18472024-11-19T01:46:33.661ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18482024-11-19T01:46:33.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18492024-11-19T01:46:33.662ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
18502024-11-19T01:46:33.663ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18512024-11-19T01:46:33.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18522024-11-19T01:46:33.665ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18532024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18542024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18552024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
18562024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
18572024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18582024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18592024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18602024-11-19T01:46:33.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
18612024-11-19T01:46:33.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18622024-11-19T01:46:33.673ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
18632024-11-19T01:46:33.673ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
18642024-11-19T01:46:33.673ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18652024-11-19T01:46:33.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18662024-11-19T01:46:33.675ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18672024-11-19T01:46:33.676ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 141.011731ms iteration = 29 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
18682024-11-19T01:46:33.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18692024-11-19T01:46:33.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18702024-11-19T01:46:33.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18712024-11-19T01:46:33.681ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
18722024-11-19T01:46:33.681ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
18732024-11-19T01:46:33.681ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-19 01:32:33.664524 UTC dns_group = internal generation = 1
18742024-11-19T01:46:33.681ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 150.009163ms iteration = 16 status = {"generation":1}
18752024-11-19T01:46:33.693ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
18762024-11-19T01:46:33.693ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
18772024-11-19T01:46:33.693ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 161.541269ms iteration = 29 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
18782024-11-19T01:46:33.696ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
18792024-11-19T01:46:33.696ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18802024-11-19T01:46:33.696ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "41e1b7b4-432c-49c5-97e5-8bdd2c8c7d98", "content-length": "26", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
18812024-11-19T01:46:33.696ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18822024-11-19T01:46:33.698ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
18832024-11-19T01:46:33.702ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18842024-11-19T01:46:33.704ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18852024-11-19T01:46:33.704ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18862024-11-19T01:46:33.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18872024-11-19T01:46:33.707ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
18882024-11-19T01:46:33.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
18892024-11-19T01:46:33.708ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
18902024-11-19T01:46:33.708ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 0 version = 2
18912024-11-19T01:46:33.708ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18922024-11-19T01:46:33.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18932024-11-19T01:46:33.743ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
18942024-11-19T01:46:33.743ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
18952024-11-19T01:46:33.744ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b88f3b2a-bc48-4e2d-823e-13ae791e1ded", "content-length": "26", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
18962024-11-19T01:46:33.744ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
18972024-11-19T01:46:33.744ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
18982024-11-19T01:46:33.744ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 212.470679ms iteration = 15 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
18992024-11-19T01:46:33.746ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
19002024-11-19T01:46:33.781ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 249.728165ms iteration = 33 status = {"physical_disks_added":0}
19012024-11-19T01:46:33.781ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
19022024-11-19T01:46:33.839ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
19032024-11-19T01:46:33.839ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19042024-11-19T01:46:33.839ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "aef7633c-2c8a-4a76-b75f-9d04d36aba78", "content-length": "372", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19052024-11-19T01:46:33.839ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19062024-11-19T01:46:33.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19072024-11-19T01:46:33.841ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19082024-11-19T01:46:33.841ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19092024-11-19T01:46:33.842ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
19102024-11-19T01:46:33.842ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
19112024-11-19T01:46:33.842ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19122024-11-19T01:46:33.843ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed80cffe-4cc5-40ad-a72b-c0ae3c77af8b", "content-length": "26", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19132024-11-19T01:46:33.843ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19142024-11-19T01:46:33.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19152024-11-19T01:46:33.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19162024-11-19T01:46:33.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19172024-11-19T01:46:33.845ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
19182024-11-19T01:46:33.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19192024-11-19T01:46:33.845ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19202024-11-19T01:46:33.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19212024-11-19T01:46:33.847ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 1 version = 2
19222024-11-19T01:46:33.847ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
19232024-11-19T01:46:33.847ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-19 01:32:36.054328 UTC dns_group = external generation = 2
19242024-11-19T01:46:33.847ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 316.222234ms iteration = 16 status = {"generation":2}
19252024-11-19T01:46:33.873ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 342.069647ms iteration = 29 status = {}
19262024-11-19T01:46:33.874ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-19T01:32:28.835276Z, time_modified: 2024-11-19T01:32:28.835276Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }) }
19272024-11-19T01:46:33.874ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 342.878181ms iteration = 16 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]}},"default_endpoint":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
19282024-11-19T01:46:33.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19292024-11-19T01:46:33.887ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
19302024-11-19T01:46:33.887ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19312024-11-19T01:46:33.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
19322024-11-19T01:46:33.890ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
19332024-11-19T01:46:33.890ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
19342024-11-19T01:46:33.890ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "64cb3809-8287-40e3-9318-030e356d0a39", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19352024-11-19T01:46:33.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19362024-11-19T01:46:33.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19372024-11-19T01:46:33.892ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19382024-11-19T01:46:33.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19392024-11-19T01:46:33.895ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
19402024-11-19T01:46:33.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
19412024-11-19T01:46:33.899ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
19422024-11-19T01:46:33.899ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
19432024-11-19T01:46:33.899ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "14a7112f-67e4-496a-acc9-cb3ddb5b40a6", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19442024-11-19T01:46:33.901ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 369.330493ms iteration = 31 status = {}
19452024-11-19T01:46:33.930ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
19462024-11-19T01:46:33.930ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "6084de26-b5ac-4516-bc5a-1ab7d1623748", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19472024-11-19T01:46:33.930ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19482024-11-19T01:46:33.932ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "60cfc326-102a-42a5-bc14-b072ab51be5d", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19492024-11-19T01:46:33.932ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19502024-11-19T01:46:33.933ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
19512024-11-19T01:46:33.933ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 402.643779ms iteration = 17 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
19522024-11-19T01:46:33.952ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
19532024-11-19T01:46:33.952ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "2b3dc305-333e-4adf-aa55-79a06e68e153", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19542024-11-19T01:46:33.952ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19552024-11-19T01:46:33.953ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
19562024-11-19T01:46:33.953ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d39bd95-7e07-4bec-a7fa-aa7ef11a2643", "content-length": "2", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19572024-11-19T01:46:33.953ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "78835dcd-ab24-4d72-9d45-790baba50f8f", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19582024-11-19T01:46:33.953ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19592024-11-19T01:46:33.956ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
19602024-11-19T01:46:33.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
19612024-11-19T01:46:33.956ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "865e37ca-3118-4893-a548-1eacfbe24ac4", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19622024-11-19T01:46:33.956ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19632024-11-19T01:46:33.958ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 427.012045ms iteration = 29 status = null
19642024-11-19T01:46:33.958ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
19652024-11-19T01:46:33.958ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 427.507574ms iteration = 17 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
19662024-11-19T01:46:34.075ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
19672024-11-19T01:46:34.075ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fc11cf9-644e-45e6-9233-dc47c67f8bb0", "content-length": "691", "date": "Tue, 19 Nov 2024 01:46:34 GMT"} })
19682024-11-19T01:46:34.075ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
19692024-11-19T01:46:34.075ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
19702024-11-19T01:46:34.075ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
19712024-11-19T01:46:34.076ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 545.335271ms iteration = 30 status = {}
19722024-11-19T01:46:34.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19732024-11-19T01:46:34.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19742024-11-19T01:46:34.090ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
19752024-11-19T01:46:34.090ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19762024-11-19T01:46:34.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19772024-11-19T01:46:34.098ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
19782024-11-19T01:46:34.099ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ff1428a-0361-49a7-bd9a-2b08194aa43f", "content-length": "134", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19792024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
19802024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
19812024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
19822024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
19832024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19842024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19852024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19862024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19872024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
19882024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
19892024-11-19T01:46:34.099ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
19902024-11-19T01:46:34.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19912024-11-19T01:46:34.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19922024-11-19T01:46:34.102ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
19932024-11-19T01:46:34.102ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebcf6dbd-9ade-4f96-8288-e2462b2c25b9", "content-length": "2", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19942024-11-19T01:46:34.102ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
19952024-11-19T01:46:34.102ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2038a78-b405-458c-9e3c-6287114d6133", "content-length": "51", "date": "Tue, 19 Nov 2024 01:46:33 GMT"} })
19962024-11-19T01:46:34.102ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
19972024-11-19T01:46:34.102ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
19982024-11-19T01:46:34.204ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "746e392d-3812-4876-bbae-5237a52c127f", "date": "Tue, 19 Nov 2024 01:46:34 GMT"} })
19992024-11-19T01:46:34.204ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
20002024-11-19T01:46:34.204ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
20012024-11-19T01:46:34.204ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d856b0f-1721-4a38-9c6a-22483c3ebaa6", "content-length": "682", "date": "Tue, 19 Nov 2024 01:46:34 GMT"} })
20022024-11-19T01:46:34.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20032024-11-19T01:46:34.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20042024-11-19T01:46:34.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20052024-11-19T01:46:34.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20062024-11-19T01:46:34.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20072024-11-19T01:46:34.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20082024-11-19T01:46:34.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20092024-11-19T01:46:34.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20102024-11-19T01:46:34.223ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 691.648078ms iteration = 29 status = {}
20112024-11-19T01:46:34.503ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a0b3fd47-a145-4b80-8d9f-e2024e83000a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20122024-11-19T01:46:34.503ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a0b3fd47-a145-4b80-8d9f-e2024e83000a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20132024-11-19T01:46:34.505ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2462 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a0b3fd47-a145-4b80-8d9f-e2024e83000a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20142024-11-19T01:46:35.530ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 024759f2-0a98-4e2d-9ff2-fc13fb3a4749 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20152024-11-19T01:46:35.530ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 024759f2-0a98-4e2d-9ff2-fc13fb3a4749 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20162024-11-19T01:46:35.532ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2735 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 024759f2-0a98-4e2d-9ff2-fc13fb3a4749 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20172024-11-19T01:46:36.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 761fc81c-4ff1-4e1a-abe0-54a2687688be roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
20182024-11-19T01:46:36.151ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 761fc81c-4ff1-4e1a-abe0-54a2687688be resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
20192024-11-19T01:46:36.154ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3448 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 761fc81c-4ff1-4e1a-abe0-54a2687688be response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
20202024-11-19T01:46:36.552ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05540a36-7061-4081-a853-67eb6cd87041 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20212024-11-19T01:46:36.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05540a36-7061-4081-a853-67eb6cd87041 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20222024-11-19T01:46:36.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2204 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 05540a36-7061-4081-a853-67eb6cd87041 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20232024-11-19T01:46:37.576ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1f43a549-1008-4fdc-8e45-92382025cced roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20242024-11-19T01:46:37.576ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1f43a549-1008-4fdc-8e45-92382025cced resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20252024-11-19T01:46:37.579ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3152 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1f43a549-1008-4fdc-8e45-92382025cced response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20262024-11-19T01:46:38.601ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5f2346de-0fc3-4918-993b-3cead74e72a3 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20272024-11-19T01:46:38.601ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5f2346de-0fc3-4918-993b-3cead74e72a3 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20282024-11-19T01:46:38.603ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2244 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5f2346de-0fc3-4918-993b-3cead74e72a3 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20292024-11-19T01:46:39.624ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d8eda6e-e435-47ab-a609-d7af1073ae39 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20302024-11-19T01:46:39.625ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d8eda6e-e435-47ab-a609-d7af1073ae39 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20312024-11-19T01:46:39.627ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2410 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4d8eda6e-e435-47ab-a609-d7af1073ae39 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20322024-11-19T01:46:40.647ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 15a5c319-4906-43ee-a9c0-63b51c83d355 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20332024-11-19T01:46:40.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 15a5c319-4906-43ee-a9c0-63b51c83d355 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20342024-11-19T01:46:40.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2902 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 15a5c319-4906-43ee-a9c0-63b51c83d355 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20352024-11-19T01:46:41.671ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5d6f374a-1ae1-4967-984c-9debb7ddcda6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20362024-11-19T01:46:41.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5d6f374a-1ae1-4967-984c-9debb7ddcda6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20372024-11-19T01:46:41.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2547 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5d6f374a-1ae1-4967-984c-9debb7ddcda6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20382024-11-19T01:46:41.812ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1867 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 507e8a4d-3d8f-47e8-bab4-1a41dd0a8f87 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
20392024-11-19T01:46:42.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0c4dda3b-9ac8-4dd0-823f-640d6383512b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20402024-11-19T01:46:42.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0c4dda3b-9ac8-4dd0-823f-640d6383512b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20412024-11-19T01:46:42.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2674 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0c4dda3b-9ac8-4dd0-823f-640d6383512b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20422024-11-19T01:46:43.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 86 reason = Timeout
20432024-11-19T01:46:43.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20442024-11-19T01:46:43.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20452024-11-19T01:46:43.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 86 reason = Timeout
20462024-11-19T01:46:43.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20472024-11-19T01:46:43.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20482024-11-19T01:46:43.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20492024-11-19T01:46:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20502024-11-19T01:46:43.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20512024-11-19T01:46:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20522024-11-19T01:46:43.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20532024-11-19T01:46:43.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20542024-11-19T01:46:43.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20552024-11-19T01:46:43.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
20562024-11-19T01:46:43.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20572024-11-19T01:46:43.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20582024-11-19T01:46:43.553ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
20592024-11-19T01:46:43.553ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.166254ms iteration = 86 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
20602024-11-19T01:46:43.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20612024-11-19T01:46:43.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20622024-11-19T01:46:43.588ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.03918ms iteration = 86 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20632024-11-19T01:46:43.717ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 21921654-80f9-415a-8bb9-0df39a7808c6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20642024-11-19T01:46:43.717ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 21921654-80f9-415a-8bb9-0df39a7808c6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20652024-11-19T01:46:43.719ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 21921654-80f9-415a-8bb9-0df39a7808c6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20662024-11-19T01:46:44.741ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3a7dc65e-6ea5-4ce0-be2d-f5dd56e7b0f3 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20672024-11-19T01:46:44.741ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3a7dc65e-6ea5-4ce0-be2d-f5dd56e7b0f3 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20682024-11-19T01:46:44.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2408 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3a7dc65e-6ea5-4ce0-be2d-f5dd56e7b0f3 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20692024-11-19T01:46:45.764ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6758acfa-2f44-42bc-bd5e-7d9f2d0838a7 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20702024-11-19T01:46:45.765ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6758acfa-2f44-42bc-bd5e-7d9f2d0838a7 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20712024-11-19T01:46:45.766ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2363 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6758acfa-2f44-42bc-bd5e-7d9f2d0838a7 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20722024-11-19T01:46:46.789ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3952592c-229f-495a-9515-a47b581b998f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20732024-11-19T01:46:46.789ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3952592c-229f-495a-9515-a47b581b998f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20742024-11-19T01:46:46.791ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2455 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3952592c-229f-495a-9515-a47b581b998f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20752024-11-19T01:46:47.812ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ad46917c-67ed-4ef4-b826-ff7422bd7dc8 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20762024-11-19T01:46:47.813ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ad46917c-67ed-4ef4-b826-ff7422bd7dc8 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20772024-11-19T01:46:47.814ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1900 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ad46917c-67ed-4ef4-b826-ff7422bd7dc8 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20782024-11-19T01:46:48.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2c0498b0-1ac5-4ea2-b69b-9db5e82a071d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20792024-11-19T01:46:48.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2c0498b0-1ac5-4ea2-b69b-9db5e82a071d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20802024-11-19T01:46:48.840ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4171 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2c0498b0-1ac5-4ea2-b69b-9db5e82a071d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20812024-11-19T01:46:49.860ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5df569be-bcd1-4f9c-88c0-96121fcca78c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20822024-11-19T01:46:49.861ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5df569be-bcd1-4f9c-88c0-96121fcca78c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20832024-11-19T01:46:49.863ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3062 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5df569be-bcd1-4f9c-88c0-96121fcca78c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20842024-11-19T01:46:50.884ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 82a8993a-fa7b-4ac5-80d3-77b167c2f2bc roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20852024-11-19T01:46:50.884ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 82a8993a-fa7b-4ac5-80d3-77b167c2f2bc resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20862024-11-19T01:46:50.887ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2929 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 82a8993a-fa7b-4ac5-80d3-77b167c2f2bc response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20872024-11-19T01:46:51.910ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3aa9d4c2-29f3-41a5-a1a3-f0f7526841fa roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20882024-11-19T01:46:51.910ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3aa9d4c2-29f3-41a5-a1a3-f0f7526841fa resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20892024-11-19T01:46:51.912ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2484 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3aa9d4c2-29f3-41a5-a1a3-f0f7526841fa response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20902024-11-19T01:46:52.935ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 50ae070f-10e5-4a44-99d0-306007d54eaa roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20912024-11-19T01:46:52.935ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 50ae070f-10e5-4a44-99d0-306007d54eaa resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20922024-11-19T01:46:52.937ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2468 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 50ae070f-10e5-4a44-99d0-306007d54eaa response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
20932024-11-19T01:46:53.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 87 reason = Timeout
20942024-11-19T01:46:53.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20952024-11-19T01:46:53.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
20962024-11-19T01:46:53.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 87 reason = Timeout
20972024-11-19T01:46:53.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20982024-11-19T01:46:53.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20992024-11-19T01:46:53.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21002024-11-19T01:46:53.559ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21012024-11-19T01:46:53.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21022024-11-19T01:46:53.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
21032024-11-19T01:46:53.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21042024-11-19T01:46:53.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21052024-11-19T01:46:53.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21062024-11-19T01:46:53.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21072024-11-19T01:46:53.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21082024-11-19T01:46:53.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
21092024-11-19T01:46:53.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21102024-11-19T01:46:53.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21112024-11-19T01:46:53.578ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
21122024-11-19T01:46:53.579ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.016784ms iteration = 87 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
21132024-11-19T01:46:53.588ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.07552ms iteration = 87 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21142024-11-19T01:46:53.961ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 53e179e3-cb0c-4af7-b0e1-8ed465c9ae78 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21152024-11-19T01:46:53.961ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 53e179e3-cb0c-4af7-b0e1-8ed465c9ae78 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21162024-11-19T01:46:53.963ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2177 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 53e179e3-cb0c-4af7-b0e1-8ed465c9ae78 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21172024-11-19T01:46:54.985ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 834898fe-88ed-418e-bff6-3d918af5eb61 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21182024-11-19T01:46:54.985ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 834898fe-88ed-418e-bff6-3d918af5eb61 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21192024-11-19T01:46:54.987ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2360 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 834898fe-88ed-418e-bff6-3d918af5eb61 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21202024-11-19T01:46:56.008ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfdc268f-8afc-4457-bf61-9d23972d4586 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21212024-11-19T01:46:56.009ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfdc268f-8afc-4457-bf61-9d23972d4586 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21222024-11-19T01:46:56.011ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2608 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bfdc268f-8afc-4457-bf61-9d23972d4586 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21232024-11-19T01:46:56.809ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1395 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = fa51a9b3-6db5-40d5-95a8-b32fe47c96b4 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
21242024-11-19T01:46:57.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7237aad6-159d-44c0-a8c2-8e47859d88f6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21252024-11-19T01:46:57.031ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7237aad6-159d-44c0-a8c2-8e47859d88f6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21262024-11-19T01:46:57.033ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2391 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7237aad6-159d-44c0-a8c2-8e47859d88f6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21272024-11-19T01:46:58.056ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 40790b2d-0a2d-4060-a856-88b844517806 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21282024-11-19T01:46:58.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 40790b2d-0a2d-4060-a856-88b844517806 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21292024-11-19T01:46:58.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2185 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 40790b2d-0a2d-4060-a856-88b844517806 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21302024-11-19T01:46:59.080ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 63dcae95-60be-40b5-80a0-3d6293ecb14b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21312024-11-19T01:46:59.080ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 63dcae95-60be-40b5-80a0-3d6293ecb14b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21322024-11-19T01:46:59.082ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2392 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 63dcae95-60be-40b5-80a0-3d6293ecb14b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21332024-11-19T01:47:00.104ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1fce98f3-b1ea-4e0b-be28-b296ba057a22 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21342024-11-19T01:47:00.105ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1fce98f3-b1ea-4e0b-be28-b296ba057a22 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21352024-11-19T01:47:00.106ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2340 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1fce98f3-b1ea-4e0b-be28-b296ba057a22 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21362024-11-19T01:47:01.128ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5c6c86e3-59c6-48f5-910e-58585e178b62 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21372024-11-19T01:47:01.128ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5c6c86e3-59c6-48f5-910e-58585e178b62 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21382024-11-19T01:47:01.130ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2079 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5c6c86e3-59c6-48f5-910e-58585e178b62 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21392024-11-19T01:47:02.151ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bc8e520c-70a6-4dd0-b41e-6959220d7ae4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21402024-11-19T01:47:02.152ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bc8e520c-70a6-4dd0-b41e-6959220d7ae4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21412024-11-19T01:47:02.153ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2066 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = bc8e520c-70a6-4dd0-b41e-6959220d7ae4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21422024-11-19T01:47:03.176ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aa6d4306-1fbd-4980-9502-c4928519299a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21432024-11-19T01:47:03.176ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aa6d4306-1fbd-4980-9502-c4928519299a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21442024-11-19T01:47:03.178ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2177 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aa6d4306-1fbd-4980-9502-c4928519299a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
21452024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 30 reason = Timeout
21462024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
21472024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 88 reason = Timeout
21482024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 30 reason = Timeout
21492024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21502024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 30 reason = Timeout
21512024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
21522024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 30 reason = Timeout
21532024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
21542024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21552024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 31 reason = Timeout
21562024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21572024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21582024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 30 reason = Timeout
21592024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21602024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 30 reason = Timeout
21612024-11-19T01:47:03.529ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21622024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
21632024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21642024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
21652024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
21662024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
21672024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
21682024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21692024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21702024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 88 reason = Timeout
21712024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Timeout
21722024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21732024-11-19T01:47:03.530ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
21742024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 30 reason = Timeout
21752024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21762024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
21772024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 30 reason = Timeout
21782024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21792024-11-19T01:47:03.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21802024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21812024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
21822024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
21832024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21842024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 30 reason = Timeout
21852024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21862024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 30 reason = Timeout
21872024-11-19T01:47:03.531ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21882024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 30 reason = Timeout
21892024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 30 reason = Timeout
21902024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
21912024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
21922024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21932024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21942024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
21952024-11-19T01:47:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
21962024-11-19T01:47:03.532ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 700258223 } })
21972024-11-19T01:47:03.532ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
21982024-11-19T01:47:03.533ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.967431ms iteration = 30 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
21992024-11-19T01:47:03.533ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 701673908 } })
22002024-11-19T01:47:03.534ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
22012024-11-19T01:47:03.534ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22022024-11-19T01:47:03.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22032024-11-19T01:47:03.535ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22042024-11-19T01:47:03.535ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22052024-11-19T01:47:03.537ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22062024-11-19T01:47:03.537ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
22072024-11-19T01:47:03.537ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22082024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22092024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
22102024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
22112024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22122024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22132024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
22142024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22152024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22162024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22172024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
22182024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22192024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22202024-11-19T01:47:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22212024-11-19T01:47:03.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22222024-11-19T01:47:03.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22232024-11-19T01:47:03.545ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22242024-11-19T01:47:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
22252024-11-19T01:47:03.551ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22262024-11-19T01:47:03.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22272024-11-19T01:47:03.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22282024-11-19T01:47:03.571ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22292024-11-19T01:47:03.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22302024-11-19T01:47:03.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22312024-11-19T01:47:03.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22322024-11-19T01:47:03.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
22332024-11-19T01:47:03.575ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22342024-11-19T01:47:03.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22352024-11-19T01:47:03.576ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22362024-11-19T01:47:03.576ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
22372024-11-19T01:47:03.576ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22382024-11-19T01:47:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
22392024-11-19T01:47:03.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22402024-11-19T01:47:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22412024-11-19T01:47:03.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22422024-11-19T01:47:03.577ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22432024-11-19T01:47:03.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22442024-11-19T01:47:03.578ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1873, tv_nsec: 746391388 } })
22452024-11-19T01:47:03.578ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
22462024-11-19T01:47:03.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
22472024-11-19T01:47:03.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22482024-11-19T01:47:03.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
22492024-11-19T01:47:03.582ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22502024-11-19T01:47:03.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22512024-11-19T01:47:03.583ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22522024-11-19T01:47:03.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
22532024-11-19T01:47:03.584ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22542024-11-19T01:47:03.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22552024-11-19T01:47:03.585ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22562024-11-19T01:47:03.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22572024-11-19T01:47:03.586ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22582024-11-19T01:47:03.587ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
22592024-11-19T01:47:03.587ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 56.121474ms iteration = 30 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
22602024-11-19T01:47:03.588ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22612024-11-19T01:47:03.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22622024-11-19T01:47:03.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
22632024-11-19T01:47:03.588ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22642024-11-19T01:47:03.589ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 57.0082ms iteration = 30 status = {"errors":[],"garbage_collect_requested":[]}
22652024-11-19T01:47:03.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22662024-11-19T01:47:03.590ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22672024-11-19T01:47:03.590ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 61.071342ms iteration = 30 status = 0
22682024-11-19T01:47:03.591ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 59.176238ms iteration = 30 status = {"errors":[],"records_set_to_done":[]}
22692024-11-19T01:47:03.591ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
22702024-11-19T01:47:03.591ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 62.004249ms iteration = 88 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
22712024-11-19T01:47:03.592ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
22722024-11-19T01:47:03.592ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22732024-11-19T01:47:03.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22742024-11-19T01:47:03.601ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22752024-11-19T01:47:03.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22762024-11-19T01:47:03.603ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22772024-11-19T01:47:03.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22782024-11-19T01:47:03.609ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
22792024-11-19T01:47:03.609ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22802024-11-19T01:47:03.610ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e45d3494-bd34-405f-be98-f97d3bbf9fd1", "content-length": "26", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
22812024-11-19T01:47:03.610ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22822024-11-19T01:47:03.611ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 80.736554ms iteration = 88 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22832024-11-19T01:47:03.612ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22842024-11-19T01:47:03.615ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
22852024-11-19T01:47:03.615ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22862024-11-19T01:47:03.615ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5087b461-b026-432e-998c-ec34a2d41c0d", "content-length": "26", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
22872024-11-19T01:47:03.615ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22882024-11-19T01:47:03.617ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22892024-11-19T01:47:03.621ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22902024-11-19T01:47:03.621ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
22912024-11-19T01:47:03.621ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 90.179032ms iteration = 30 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
22922024-11-19T01:47:03.626ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22932024-11-19T01:47:03.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22942024-11-19T01:47:03.628ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 98.369739ms iteration = 30 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
22952024-11-19T01:47:03.633ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 101.698958ms iteration = 30 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
22962024-11-19T01:47:03.666ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 134.2028ms iteration = 30 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
22972024-11-19T01:47:03.669ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
22982024-11-19T01:47:03.669ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22992024-11-19T01:47:03.669ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eddfb5a5-7ade-4055-89f1-2edfe279f02c", "content-length": "26", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23002024-11-19T01:47:03.669ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23012024-11-19T01:47:03.671ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
23022024-11-19T01:47:03.677ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23032024-11-19T01:47:03.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23042024-11-19T01:47:03.679ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23052024-11-19T01:47:03.681ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23062024-11-19T01:47:03.681ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23072024-11-19T01:47:03.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23082024-11-19T01:47:03.683ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23092024-11-19T01:47:03.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23102024-11-19T01:47:03.684ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23112024-11-19T01:47:03.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
23122024-11-19T01:47:03.717ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
23132024-11-19T01:47:03.717ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "0996888d-0e20-46c4-b5b9-b622aecfb8e6", "content-length": "2", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23142024-11-19T01:47:03.719ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
23152024-11-19T01:47:03.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
23162024-11-19T01:47:03.722ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 192.266747ms iteration = 30 status = null
23172024-11-19T01:47:03.724ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 194.531178ms iteration = 30 status = {}
23182024-11-19T01:47:03.727ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 198.052401ms iteration = 34 status = {"physical_disks_added":0}
23192024-11-19T01:47:03.773ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
23202024-11-19T01:47:03.773ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
23212024-11-19T01:47:03.773ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8ba06a7-ac82-4054-a5f0-8a4b0bafe558", "content-length": "372", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23222024-11-19T01:47:03.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23232024-11-19T01:47:03.773ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
23242024-11-19T01:47:03.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23252024-11-19T01:47:03.773ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ebfb025-d33c-4078-8349-d212ec0e7df8", "content-length": "691", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23262024-11-19T01:47:03.773ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
23272024-11-19T01:47:03.773ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
23282024-11-19T01:47:03.773ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
23292024-11-19T01:47:03.775ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 245.203484ms iteration = 31 status = {}
23302024-11-19T01:47:03.820ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
23312024-11-19T01:47:03.820ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
23322024-11-19T01:47:03.828ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23332024-11-19T01:47:03.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23342024-11-19T01:47:03.831ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
23352024-11-19T01:47:03.831ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23362024-11-19T01:47:03.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23372024-11-19T01:47:03.841ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
23382024-11-19T01:47:03.841ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "266aafd0-dc94-49bd-bcac-249f2edaf595", "content-length": "134", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23392024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
23402024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
23412024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
23422024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23432024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23442024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23452024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23462024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23472024-11-19T01:47:03.841ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23482024-11-19T01:47:03.842ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23492024-11-19T01:47:03.842ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
23502024-11-19T01:47:03.842ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23512024-11-19T01:47:03.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23522024-11-19T01:47:03.844ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
23532024-11-19T01:47:03.844ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "e12db6d4-67b1-49bf-8d7c-c2143f379d3c", "content-length": "2", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23542024-11-19T01:47:03.844ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
23552024-11-19T01:47:03.844ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "4db3ae0c-1f72-4760-b4f7-c4f3b71bbb26", "content-length": "51", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23562024-11-19T01:47:03.844ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
23572024-11-19T01:47:03.844ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23582024-11-19T01:47:03.893ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "56deed41-c045-48ed-b3be-b59469d780ae", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23592024-11-19T01:47:03.893ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23602024-11-19T01:47:03.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23612024-11-19T01:47:03.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23622024-11-19T01:47:03.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23632024-11-19T01:47:03.899ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23642024-11-19T01:47:03.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23652024-11-19T01:47:03.902ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
23662024-11-19T01:47:03.902ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
23672024-11-19T01:47:03.903ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6ef13d56-addd-4344-861b-e691bd9bbaeb", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23682024-11-19T01:47:03.904ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 373.806804ms iteration = 32 status = {}
23692024-11-19T01:47:03.993ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "62ffaf8c-8d43-4ff9-a69f-233871eff113", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23702024-11-19T01:47:03.993ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
23712024-11-19T01:47:03.993ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23722024-11-19T01:47:03.993ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2662035-1a85-457f-8e86-944eaeefb09f", "content-length": "682", "date": "Tue, 19 Nov 2024 01:47:03 GMT"} })
23732024-11-19T01:47:03.993ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23742024-11-19T01:47:03.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23752024-11-19T01:47:03.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23762024-11-19T01:47:03.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23772024-11-19T01:47:04.001ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23782024-11-19T01:47:04.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23792024-11-19T01:47:04.003ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23802024-11-19T01:47:04.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23812024-11-19T01:47:04.011ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 481.328706ms iteration = 30 status = {}
23822024-11-19T01:47:04.199ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 264b4492-4ac1-4e6f-93fb-be395f246bb7 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23832024-11-19T01:47:04.199ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 264b4492-4ac1-4e6f-93fb-be395f246bb7 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23842024-11-19T01:47:04.201ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2075 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 264b4492-4ac1-4e6f-93fb-be395f246bb7 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23852024-11-19T01:47:05.221ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c7c4ba1a-210e-4500-91ca-b333f868b65e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23862024-11-19T01:47:05.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c7c4ba1a-210e-4500-91ca-b333f868b65e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23872024-11-19T01:47:05.223ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2159 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c7c4ba1a-210e-4500-91ca-b333f868b65e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23882024-11-19T01:47:06.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = f1e860f9-9792-48a1-b873-7906e0b334d0 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
23892024-11-19T01:47:06.184ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = f1e860f9-9792-48a1-b873-7906e0b334d0 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
23902024-11-19T01:47:06.186ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2704 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = f1e860f9-9792-48a1-b873-7906e0b334d0 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
23912024-11-19T01:47:06.244ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 80e97fcf-f165-464d-9268-a0347e89df81 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23922024-11-19T01:47:06.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 80e97fcf-f165-464d-9268-a0347e89df81 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23932024-11-19T01:47:06.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2380 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 80e97fcf-f165-464d-9268-a0347e89df81 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23942024-11-19T01:47:07.270ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d59535b7-406a-48b9-9f97-8e1cf2871cf9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23952024-11-19T01:47:07.270ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d59535b7-406a-48b9-9f97-8e1cf2871cf9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23962024-11-19T01:47:07.272ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2018 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d59535b7-406a-48b9-9f97-8e1cf2871cf9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23972024-11-19T01:47:08.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd907541-13a2-4862-9d8b-c24f5717a7c2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23982024-11-19T01:47:08.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd907541-13a2-4862-9d8b-c24f5717a7c2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
23992024-11-19T01:47:08.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2287 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd907541-13a2-4862-9d8b-c24f5717a7c2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24002024-11-19T01:47:09.318ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2fe79276-ed68-4746-8abd-54cb1074493f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24012024-11-19T01:47:09.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2fe79276-ed68-4746-8abd-54cb1074493f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24022024-11-19T01:47:09.320ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2260 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2fe79276-ed68-4746-8abd-54cb1074493f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24032024-11-19T01:47:10.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c0edd77-6a95-48cb-87cc-127f1d2ccc7b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24042024-11-19T01:47:10.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c0edd77-6a95-48cb-87cc-127f1d2ccc7b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24052024-11-19T01:47:10.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2737 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8c0edd77-6a95-48cb-87cc-127f1d2ccc7b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24062024-11-19T01:47:11.366ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4175bda0-64fc-4921-a46c-e632b1cdb8ef roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24072024-11-19T01:47:11.367ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4175bda0-64fc-4921-a46c-e632b1cdb8ef resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24082024-11-19T01:47:11.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2279 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4175bda0-64fc-4921-a46c-e632b1cdb8ef response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24092024-11-19T01:47:11.808ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1398 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = e9bd216d-3a4d-47b0-bfe7-164e933984e0 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
24102024-11-19T01:47:12.392ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1e2d57cf-c2c9-4c7c-869f-1b6339032862 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24112024-11-19T01:47:12.392ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1e2d57cf-c2c9-4c7c-869f-1b6339032862 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24122024-11-19T01:47:12.393ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1959 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1e2d57cf-c2c9-4c7c-869f-1b6339032862 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24132024-11-19T01:47:13.416ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b3f096fb-7f84-4853-96ed-02a67d5e436e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24142024-11-19T01:47:13.417ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b3f096fb-7f84-4853-96ed-02a67d5e436e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24152024-11-19T01:47:13.419ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2795 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b3f096fb-7f84-4853-96ed-02a67d5e436e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24162024-11-19T01:47:13.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 89 reason = Timeout
24172024-11-19T01:47:13.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24182024-11-19T01:47:13.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24192024-11-19T01:47:13.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24202024-11-19T01:47:13.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 89 reason = Timeout
24212024-11-19T01:47:13.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24222024-11-19T01:47:13.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24232024-11-19T01:47:13.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24242024-11-19T01:47:13.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24252024-11-19T01:47:13.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24262024-11-19T01:47:13.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24272024-11-19T01:47:13.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24282024-11-19T01:47:13.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24292024-11-19T01:47:13.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24302024-11-19T01:47:13.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24312024-11-19T01:47:13.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
24322024-11-19T01:47:13.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24332024-11-19T01:47:13.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24342024-11-19T01:47:13.550ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
24352024-11-19T01:47:13.550ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.159259ms iteration = 89 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
24362024-11-19T01:47:13.560ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 29.424267ms iteration = 89 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24372024-11-19T01:47:14.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 74029e52-aaa3-4981-85c5-2690e01b05ab roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24382024-11-19T01:47:14.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 74029e52-aaa3-4981-85c5-2690e01b05ab resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24392024-11-19T01:47:14.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2425 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 74029e52-aaa3-4981-85c5-2690e01b05ab response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24402024-11-19T01:47:15.464ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aeda1279-065c-412c-a7d0-8f0157b15bb2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24412024-11-19T01:47:15.464ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aeda1279-065c-412c-a7d0-8f0157b15bb2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24422024-11-19T01:47:15.466ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2686 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aeda1279-065c-412c-a7d0-8f0157b15bb2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24432024-11-19T01:47:16.489ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3dbe9fa1-a2d6-4025-98bc-a0e1aed3003b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24442024-11-19T01:47:16.490ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3dbe9fa1-a2d6-4025-98bc-a0e1aed3003b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24452024-11-19T01:47:16.491ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2022 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3dbe9fa1-a2d6-4025-98bc-a0e1aed3003b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24462024-11-19T01:47:16.801ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 03f35fe1-2df2-4482-9a59-5dca945e77fb roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
24472024-11-19T01:47:16.801ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 03f35fe1-2df2-4482-9a59-5dca945e77fb resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
24482024-11-19T01:47:16.804ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3727 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 03f35fe1-2df2-4482-9a59-5dca945e77fb response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?sort_by=id_ascending
24492024-11-19T01:47:16.804ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 29594fa3-0728-4109-a052-47207454ab09 roles = RoleSet { roles: {} } uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
24502024-11-19T01:47:16.805ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 29594fa3-0728-4109-a052-47207454ab09 resource = Database result = Ok(()) uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
24512024-11-19T01:47:16.807ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2469 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46271 req_id = 29594fa3-0728-4109-a052-47207454ab09 response_code = 200 uri = /metrics/collectors/b8403c47-1584-4cc7-8521-f7e7869f6be7/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNzkxZWQ4OS0xMzM1LTRlODAtOTA0NS04ZjFjNTA2NTQ4OWYifX0%3D
24522024-11-19T01:47:17.514ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c4297f38-5ca7-4954-b81a-cd83c199d4c4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24532024-11-19T01:47:17.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c4297f38-5ca7-4954-b81a-cd83c199d4c4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24542024-11-19T01:47:17.517ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2624 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c4297f38-5ca7-4954-b81a-cd83c199d4c4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24552024-11-19T01:47:18.539ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7967a1f9-7e2c-4ef8-a61c-1a10e6c1702c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24562024-11-19T01:47:18.539ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7967a1f9-7e2c-4ef8-a61c-1a10e6c1702c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24572024-11-19T01:47:18.541ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2398 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7967a1f9-7e2c-4ef8-a61c-1a10e6c1702c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24582024-11-19T01:47:19.563ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0055b49c-a06a-4d68-862b-f018fa4ab666 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24592024-11-19T01:47:19.563ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0055b49c-a06a-4d68-862b-f018fa4ab666 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24602024-11-19T01:47:19.565ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1791 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0055b49c-a06a-4d68-862b-f018fa4ab666 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24612024-11-19T01:47:20.586ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 470c651f-7ddd-4a48-ab56-0ffe651748fa roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24622024-11-19T01:47:20.587ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 470c651f-7ddd-4a48-ab56-0ffe651748fa resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24632024-11-19T01:47:20.589ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2765 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 470c651f-7ddd-4a48-ab56-0ffe651748fa response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24642024-11-19T01:47:21.613ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7bc436c6-bdc1-489f-a10e-9cbe0c8e2e70 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24652024-11-19T01:47:21.613ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7bc436c6-bdc1-489f-a10e-9cbe0c8e2e70 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24662024-11-19T01:47:21.615ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2205 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7bc436c6-bdc1-489f-a10e-9cbe0c8e2e70 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24672024-11-19T01:47:22.639ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2755b21e-44bc-48d4-b30e-1243141c89cc roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24682024-11-19T01:47:22.640ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2755b21e-44bc-48d4-b30e-1243141c89cc resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24692024-11-19T01:47:22.642ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2564 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2755b21e-44bc-48d4-b30e-1243141c89cc response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24702024-11-19T01:47:23.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 90 reason = Timeout
24712024-11-19T01:47:23.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24722024-11-19T01:47:23.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24732024-11-19T01:47:23.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24742024-11-19T01:47:23.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 90 reason = Timeout
24752024-11-19T01:47:23.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24762024-11-19T01:47:23.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24772024-11-19T01:47:23.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24782024-11-19T01:47:23.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24792024-11-19T01:47:23.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24802024-11-19T01:47:23.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24812024-11-19T01:47:23.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24822024-11-19T01:47:23.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.264298ms iteration = 90 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24832024-11-19T01:47:23.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24842024-11-19T01:47:23.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24852024-11-19T01:47:23.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24862024-11-19T01:47:23.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
24872024-11-19T01:47:23.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24882024-11-19T01:47:23.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24892024-11-19T01:47:23.546ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
24902024-11-19T01:47:23.546ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 17.435102ms iteration = 90 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
24912024-11-19T01:47:23.663ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 734c2714-a842-4d49-a748-406ce68ce3b3 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24922024-11-19T01:47:23.664ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 734c2714-a842-4d49-a748-406ce68ce3b3 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24932024-11-19T01:47:23.665ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1757 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 734c2714-a842-4d49-a748-406ce68ce3b3 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24942024-11-19T01:47:24.688ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f5e083f1-e0d1-4b2d-8097-17ad2aa184e2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24952024-11-19T01:47:24.689ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f5e083f1-e0d1-4b2d-8097-17ad2aa184e2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24962024-11-19T01:47:24.691ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2576 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f5e083f1-e0d1-4b2d-8097-17ad2aa184e2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24972024-11-19T01:47:25.713ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d36e8ae2-a54e-45b6-ae46-5520fd02529f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24982024-11-19T01:47:25.713ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d36e8ae2-a54e-45b6-ae46-5520fd02529f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
24992024-11-19T01:47:25.715ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2602 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d36e8ae2-a54e-45b6-ae46-5520fd02529f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25002024-11-19T01:47:26.737ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5ac89308-3dfa-471e-ab8d-730a78843c53 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25012024-11-19T01:47:26.738ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5ac89308-3dfa-471e-ab8d-730a78843c53 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25022024-11-19T01:47:26.740ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2189 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5ac89308-3dfa-471e-ab8d-730a78843c53 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25032024-11-19T01:47:26.810ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1407 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = e67500df-248d-42ae-9b13-08e986662492 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
25042024-11-19T01:47:27.762ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 35f0ea4d-a0b8-4c9b-8a5b-e76331fad37d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25052024-11-19T01:47:27.763ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 35f0ea4d-a0b8-4c9b-8a5b-e76331fad37d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25062024-11-19T01:47:27.765ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2602 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 35f0ea4d-a0b8-4c9b-8a5b-e76331fad37d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25072024-11-19T01:47:28.788ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b12c07d9-9a71-45d2-aded-2271ec3fb633 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25082024-11-19T01:47:28.788ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b12c07d9-9a71-45d2-aded-2271ec3fb633 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25092024-11-19T01:47:28.790ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1964 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b12c07d9-9a71-45d2-aded-2271ec3fb633 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25102024-11-19T01:47:29.810ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 65c167b7-8963-43b4-b932-b7ab36f2a941 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25112024-11-19T01:47:29.811ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 65c167b7-8963-43b4-b932-b7ab36f2a941 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25122024-11-19T01:47:29.813ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2304 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 65c167b7-8963-43b4-b932-b7ab36f2a941 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25132024-11-19T01:47:30.836ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 34274220-59f3-457e-a4bd-e2f875b5c0c9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25142024-11-19T01:47:30.836ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 34274220-59f3-457e-a4bd-e2f875b5c0c9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25152024-11-19T01:47:30.838ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2711 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 34274220-59f3-457e-a4bd-e2f875b5c0c9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25162024-11-19T01:47:31.863ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5cd110b5-1182-460e-bce4-8e9af86af453 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25172024-11-19T01:47:31.863ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5cd110b5-1182-460e-bce4-8e9af86af453 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25182024-11-19T01:47:31.865ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2123 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5cd110b5-1182-460e-bce4-8e9af86af453 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25192024-11-19T01:47:32.888ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 88de78b6-fd15-42ff-930c-d96167860306 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25202024-11-19T01:47:32.888ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 88de78b6-fd15-42ff-930c-d96167860306 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25212024-11-19T01:47:32.890ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2505 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 88de78b6-fd15-42ff-930c-d96167860306 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
25222024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 16 reason = Timeout
25232024-11-19T01:47:33.529ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-19 01:37:33.529756619 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
25242024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 17 reason = Timeout
25252024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
25262024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 31 reason = Timeout
25272024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 17 reason = Timeout
25282024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 17 reason = Timeout
25292024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 18 reason = Timeout
25302024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25312024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
25322024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25332024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 31 reason = Timeout
25342024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 17 reason = Timeout
25352024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
25362024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 17 reason = Timeout
25372024-11-19T01:47:33.529ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
25382024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25392024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 17 reason = Timeout
25402024-11-19T01:47:33.530ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120
25412024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.86\u{b5}s iteration = 17 status = {"enabled":false,"target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120"}
25422024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 91 reason = Timeout
25432024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 18 reason = Timeout
25442024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25452024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 31 reason = Timeout
25462024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 31 reason = Timeout
25472024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25482024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
25492024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 31 reason = Timeout
25502024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25512024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 31 reason = Timeout
25522024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 32 reason = Timeout
25532024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25542024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25552024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 7 reason = Timeout
25562024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
25572024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 16 reason = Timeout
25582024-11-19T01:47:33.530ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
25592024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
25602024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Timeout
25612024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25622024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
25632024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
25642024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
25652024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25662024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25672024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25682024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25692024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
25702024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
25712024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25722024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25732024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25742024-11-19T01:47:33.530ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("52e4d19a-b928-4138-8817-4fd8deeaf29e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 698393905 } })
25752024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
25762024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25772024-11-19T01:47:33.530ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e98d393-d940-4def-ba7c-d8c67ea23aa1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e48e23d9-6a0a-4bc6-9e0b-f83ba78952f1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 698396995 } })
25782024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25792024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
25802024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
25812024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
25822024-11-19T01:47:33.530ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
25832024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 91 reason = Timeout
25842024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25852024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25862024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 4 reason = Timeout
25872024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 31 reason = Timeout
25882024-11-19T01:47:33.530ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
25892024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
25902024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
25912024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25922024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 16 reason = Timeout
25932024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
25942024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 31 reason = Timeout
25952024-11-19T01:47:33.530ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
25962024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
25972024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
25982024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
25992024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.376155ms iteration = 17 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
26002024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
26012024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.330124ms iteration = 17 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
26022024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26032024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
26042024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
26052024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26062024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26072024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26082024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26092024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
26102024-11-19T01:47:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26112024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 31 reason = Timeout
26122024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 31 reason = Timeout
26132024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26142024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26152024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 31 reason = Timeout
26162024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
26172024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 16 reason = Timeout
26182024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
26192024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 16 reason = Timeout
26202024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
26212024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 31 reason = Timeout
26222024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
26232024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26242024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26252024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
26262024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26272024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
26282024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
26292024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26302024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
26312024-11-19T01:47:33.532ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-19 01:37:33.529756619 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
26322024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.096775ms iteration = 16 status = {"expiration":"2024-11-19T01:37:33.529756619Z","pruned":[]}
26332024-11-19T01:47:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26342024-11-19T01:47:33.533ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26352024-11-19T01:47:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26362024-11-19T01:47:33.533ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 701274247 } })
26372024-11-19T01:47:33.533ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
26382024-11-19T01:47:33.534ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.426839ms iteration = 31 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
26392024-11-19T01:47:33.534ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
26402024-11-19T01:47:33.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26412024-11-19T01:47:33.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26422024-11-19T01:47:33.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26432024-11-19T01:47:33.536ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26442024-11-19T01:47:33.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26452024-11-19T01:47:33.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26462024-11-19T01:47:33.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26472024-11-19T01:47:33.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26482024-11-19T01:47:33.538ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 706448239 } })
26492024-11-19T01:47:33.538ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26502024-11-19T01:47:33.539ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
26512024-11-19T01:47:33.539ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26522024-11-19T01:47:33.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
26532024-11-19T01:47:33.539ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
26542024-11-19T01:47:33.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26552024-11-19T01:47:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
26562024-11-19T01:47:33.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26572024-11-19T01:47:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26582024-11-19T01:47:33.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26592024-11-19T01:47:33.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26602024-11-19T01:47:33.541ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26612024-11-19T01:47:33.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26622024-11-19T01:47:33.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
26632024-11-19T01:47:33.542ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
26642024-11-19T01:47:33.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
26652024-11-19T01:47:33.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26662024-11-19T01:47:33.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26672024-11-19T01:47:33.543ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26682024-11-19T01:47:33.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26692024-11-19T01:47:33.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
26702024-11-19T01:47:33.544ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
26712024-11-19T01:47:33.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26722024-11-19T01:47:33.545ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26732024-11-19T01:47:33.546ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
26742024-11-19T01:47:33.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
26752024-11-19T01:47:33.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
26762024-11-19T01:47:33.546ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
26772024-11-19T01:47:33.546ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26782024-11-19T01:47:33.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26792024-11-19T01:47:33.548ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26802024-11-19T01:47:33.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26812024-11-19T01:47:33.548ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26822024-11-19T01:47:33.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26832024-11-19T01:47:33.549ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26842024-11-19T01:47:33.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26852024-11-19T01:47:33.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
26862024-11-19T01:47:33.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
26872024-11-19T01:47:33.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26882024-11-19T01:47:33.552ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26892024-11-19T01:47:33.552ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
26902024-11-19T01:47:33.552ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
26912024-11-19T01:47:33.552ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 20.381005ms iteration = 16 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
26922024-11-19T01:47:33.553ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.508274ms iteration = 16 status = {"errors":[],"found_port_ok":[]}
26932024-11-19T01:47:33.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
26942024-11-19T01:47:33.554ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
26952024-11-19T01:47:33.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
26962024-11-19T01:47:33.557ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26972024-11-19T01:47:33.559ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26982024-11-19T01:47:33.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
26992024-11-19T01:47:33.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27002024-11-19T01:47:33.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27012024-11-19T01:47:33.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27022024-11-19T01:47:33.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27032024-11-19T01:47:33.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27042024-11-19T01:47:33.566ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1903, tv_nsec: 734611603 } })
27052024-11-19T01:47:33.566ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27062024-11-19T01:47:33.568ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
27072024-11-19T01:47:33.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27082024-11-19T01:47:33.572ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 42.291376ms iteration = 7 status = {"nsuccess":5}
27092024-11-19T01:47:33.574ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.513598ms iteration = 91 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
27102024-11-19T01:47:33.574ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27112024-11-19T01:47:33.574ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
27122024-11-19T01:47:33.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
27132024-11-19T01:47:33.578ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
27142024-11-19T01:47:33.578ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27152024-11-19T01:47:33.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27162024-11-19T01:47:33.585ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
27172024-11-19T01:47:33.586ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27182024-11-19T01:47:33.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
27192024-11-19T01:47:33.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
27202024-11-19T01:47:33.588ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
27212024-11-19T01:47:33.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
27222024-11-19T01:47:33.595ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27232024-11-19T01:47:33.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27242024-11-19T01:47:33.597ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
27252024-11-19T01:47:33.597ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27262024-11-19T01:47:33.597ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
27272024-11-19T01:47:33.597ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27282024-11-19T01:47:33.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
27292024-11-19T01:47:33.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27302024-11-19T01:47:33.600ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27312024-11-19T01:47:33.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27322024-11-19T01:47:33.601ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
27332024-11-19T01:47:33.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
27342024-11-19T01:47:33.603ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27352024-11-19T01:47:33.603ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27362024-11-19T01:47:33.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27372024-11-19T01:47:33.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
27382024-11-19T01:47:33.605ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
27392024-11-19T01:47:33.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
27402024-11-19T01:47:33.605ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.605166ms iteration = 31 status = {"errors":[],"garbage_collect_requested":[]}
27412024-11-19T01:47:33.607ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.807518ms iteration = 31 status = {"errors":[],"records_set_to_done":[]}
27422024-11-19T01:47:33.607ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27432024-11-19T01:47:33.607ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
27442024-11-19T01:47:33.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
27452024-11-19T01:47:33.608ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
27462024-11-19T01:47:33.608ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
27472024-11-19T01:47:33.608ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27482024-11-19T01:47:33.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
27492024-11-19T01:47:33.609ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27502024-11-19T01:47:33.609ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27512024-11-19T01:47:33.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27522024-11-19T01:47:33.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27532024-11-19T01:47:33.611ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27542024-11-19T01:47:33.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27552024-11-19T01:47:33.613ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
27562024-11-19T01:47:33.613ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
27572024-11-19T01:47:33.613ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
27582024-11-19T01:47:33.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27592024-11-19T01:47:33.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27602024-11-19T01:47:33.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27612024-11-19T01:47:33.614ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27622024-11-19T01:47:33.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27632024-11-19T01:47:33.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27642024-11-19T01:47:33.615ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
27652024-11-19T01:47:33.615ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
27662024-11-19T01:47:33.615ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 85.883235ms iteration = 16 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
27672024-11-19T01:47:33.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
27682024-11-19T01:47:33.618ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27692024-11-19T01:47:33.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27702024-11-19T01:47:33.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27712024-11-19T01:47:33.623ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
27722024-11-19T01:47:33.623ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 92.524955ms iteration = 31 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
27732024-11-19T01:47:33.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
27742024-11-19T01:47:33.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27752024-11-19T01:47:33.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
27762024-11-19T01:47:33.636ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
27772024-11-19T01:47:33.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
27782024-11-19T01:47:33.638ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27792024-11-19T01:47:33.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27802024-11-19T01:47:33.641ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27812024-11-19T01:47:33.641ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27822024-11-19T01:47:33.641ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 109.343506ms iteration = 31 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
27832024-11-19T01:47:33.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27842024-11-19T01:47:33.643ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27852024-11-19T01:47:33.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27862024-11-19T01:47:33.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
27872024-11-19T01:47:33.643ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27882024-11-19T01:47:33.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
27892024-11-19T01:47:33.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 116.391951ms iteration = 31 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
27902024-11-19T01:47:33.646ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
27912024-11-19T01:47:33.646ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 116.87651ms iteration = 91 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
27922024-11-19T01:47:33.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27932024-11-19T01:47:33.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
27942024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
27952024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
27962024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27972024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
27982024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
27992024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
28002024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28012024-11-19T01:47:33.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28022024-11-19T01:47:33.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28032024-11-19T01:47:33.661ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28042024-11-19T01:47:33.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28052024-11-19T01:47:33.662ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28062024-11-19T01:47:33.662ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28072024-11-19T01:47:33.663ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28082024-11-19T01:47:33.663ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28092024-11-19T01:47:33.663ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
28102024-11-19T01:47:33.663ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 132.386698ms iteration = 31 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
28112024-11-19T01:47:33.663ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28122024-11-19T01:47:33.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
28132024-11-19T01:47:33.666ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28142024-11-19T01:47:33.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28152024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28162024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28172024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28182024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28192024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28202024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28212024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
28222024-11-19T01:47:33.667ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28232024-11-19T01:47:33.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28242024-11-19T01:47:33.668ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 138.166771ms iteration = 31 status = 0
28252024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
28262024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 0 version = 2
28272024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28282024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
28292024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
28302024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-19 01:32:33.664524 UTC dns_group = internal generation = 1
28312024-11-19T01:47:33.669ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 140.093865ms iteration = 17 status = {"generation":1}
28322024-11-19T01:47:33.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28332024-11-19T01:47:33.670ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28342024-11-19T01:47:33.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28352024-11-19T01:47:33.671ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28362024-11-19T01:47:33.671ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
28372024-11-19T01:47:33.672ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28382024-11-19T01:47:33.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28392024-11-19T01:47:33.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28402024-11-19T01:47:33.673ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28412024-11-19T01:47:33.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28422024-11-19T01:47:33.674ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
28432024-11-19T01:47:33.678ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28442024-11-19T01:47:33.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28452024-11-19T01:47:33.679ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 1 version = 2
28462024-11-19T01:47:33.679ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
28472024-11-19T01:47:33.679ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-19 01:32:36.054328 UTC dns_group = external generation = 2
28482024-11-19T01:47:33.679ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 149.237699ms iteration = 17 status = {"generation":2}
28492024-11-19T01:47:33.680ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28502024-11-19T01:47:33.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28512024-11-19T01:47:33.680ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
28522024-11-19T01:47:33.680ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28532024-11-19T01:47:33.680ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28542024-11-19T01:47:33.681ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 148.977074ms iteration = 31 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
28552024-11-19T01:47:33.681ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6e514a5-3fbc-42e3-ad71-4754be824c01", "content-length": "26", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
28562024-11-19T01:47:33.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28572024-11-19T01:47:33.681ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28582024-11-19T01:47:33.682ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28592024-11-19T01:47:33.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28602024-11-19T01:47:33.683ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28612024-11-19T01:47:33.684ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28622024-11-19T01:47:33.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28632024-11-19T01:47:33.687ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
28642024-11-19T01:47:33.687ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28652024-11-19T01:47:33.688ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1642aaab-a326-482b-93de-c694d542cae0", "content-length": "26", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
28662024-11-19T01:47:33.688ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28672024-11-19T01:47:33.690ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
28682024-11-19T01:47:33.698ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-19T01:32:28.835276Z, time_modified: 2024-11-19T01:32:28.835276Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }) }
28692024-11-19T01:47:33.698ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 168.31829ms iteration = 17 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]}},"default_endpoint":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
28702024-11-19T01:47:33.714ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28712024-11-19T01:47:33.717ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
28722024-11-19T01:47:33.717ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28732024-11-19T01:47:33.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28742024-11-19T01:47:33.719ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28752024-11-19T01:47:33.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28762024-11-19T01:47:33.722ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28772024-11-19T01:47:33.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28782024-11-19T01:47:33.725ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28792024-11-19T01:47:33.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28802024-11-19T01:47:33.726ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
28812024-11-19T01:47:33.726ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28822024-11-19T01:47:33.726ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c36b8c88-937d-40bb-bd38-b8b826486fd0", "content-length": "26", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
28832024-11-19T01:47:33.726ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
28842024-11-19T01:47:33.727ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28852024-11-19T01:47:33.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28862024-11-19T01:47:33.728ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
28872024-11-19T01:47:33.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
28882024-11-19T01:47:33.729ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
28892024-11-19T01:47:33.738ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
28902024-11-19T01:47:33.738ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 207.536671ms iteration = 16 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
28912024-11-19T01:47:33.758ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 228.73335ms iteration = 35 status = {"physical_disks_added":0}
28922024-11-19T01:47:33.789ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
28932024-11-19T01:47:33.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
28942024-11-19T01:47:33.810ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
28952024-11-19T01:47:33.822ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
28962024-11-19T01:47:33.823ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "07a743f6-e3b1-43d8-9825-825ff8fb61fe", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28972024-11-19T01:47:33.823ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28982024-11-19T01:47:33.825ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f5351b03-612e-4833-b66b-4b334c5d7c55", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
28992024-11-19T01:47:33.825ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
29002024-11-19T01:47:33.826ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
29012024-11-19T01:47:33.826ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 297.076173ms iteration = 18 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
29022024-11-19T01:47:33.830ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 300.527074ms iteration = 31 status = {}
29032024-11-19T01:47:33.843ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
29042024-11-19T01:47:33.843ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe5c612d-e172-4067-a9f8-09e0bfe68f7d", "content-length": "2", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
29052024-11-19T01:47:33.845ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
29062024-11-19T01:47:33.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
29072024-11-19T01:47:33.847ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
29082024-11-19T01:47:33.847ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29092024-11-19T01:47:33.847ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "57ef25ba-35b0-4d06-9206-fa074bd5bcbf", "content-length": "372", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
29102024-11-19T01:47:33.847ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29112024-11-19T01:47:33.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29122024-11-19T01:47:33.848ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29132024-11-19T01:47:33.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29142024-11-19T01:47:33.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29152024-11-19T01:47:33.850ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 320.325998ms iteration = 31 status = null
29162024-11-19T01:47:33.850ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29172024-11-19T01:47:33.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29182024-11-19T01:47:33.851ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29192024-11-19T01:47:33.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29202024-11-19T01:47:33.852ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29212024-11-19T01:47:33.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29222024-11-19T01:47:33.854ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29232024-11-19T01:47:33.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29242024-11-19T01:47:33.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
29252024-11-19T01:47:33.854ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29262024-11-19T01:47:33.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29272024-11-19T01:47:33.856ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29282024-11-19T01:47:33.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29292024-11-19T01:47:33.857ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29302024-11-19T01:47:33.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29312024-11-19T01:47:33.859ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29322024-11-19T01:47:33.862ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
29332024-11-19T01:47:33.862ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
29342024-11-19T01:47:33.862ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "55a5c0fa-515c-41e2-974a-7c54c387cd28", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
29352024-11-19T01:47:33.862ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29362024-11-19T01:47:33.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29372024-11-19T01:47:33.863ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
29382024-11-19T01:47:33.863ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29392024-11-19T01:47:33.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29402024-11-19T01:47:33.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29412024-11-19T01:47:33.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29422024-11-19T01:47:33.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29432024-11-19T01:47:33.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29442024-11-19T01:47:33.868ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29452024-11-19T01:47:33.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29462024-11-19T01:47:33.870ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29472024-11-19T01:47:33.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29482024-11-19T01:47:33.870ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
29492024-11-19T01:47:33.870ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
29502024-11-19T01:47:33.871ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ffde5ffa-bb48-4b27-8f29-6b2d04cdf470", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
29512024-11-19T01:47:33.871ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29522024-11-19T01:47:33.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29532024-11-19T01:47:33.873ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
29542024-11-19T01:47:33.873ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 342.874681ms iteration = 33 status = {}
29552024-11-19T01:47:33.873ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29562024-11-19T01:47:33.873ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a1af63f6-ec43-4936-9249-a036a036f429", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29572024-11-19T01:47:33.873ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29582024-11-19T01:47:33.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29592024-11-19T01:47:33.875ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3079f63f-d09e-457a-874c-ea8bd9ebb05f", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29602024-11-19T01:47:33.875ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29612024-11-19T01:47:33.876ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29622024-11-19T01:47:33.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29632024-11-19T01:47:33.877ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b7eaf17a-785b-4d72-bf35-e8c426f457bc", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29642024-11-19T01:47:33.877ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29652024-11-19T01:47:33.877ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29662024-11-19T01:47:33.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29672024-11-19T01:47:33.879ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29682024-11-19T01:47:33.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29692024-11-19T01:47:33.879ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
29702024-11-19T01:47:33.879ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 349.859866ms iteration = 18 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
29712024-11-19T01:47:33.880ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29722024-11-19T01:47:33.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
29732024-11-19T01:47:33.883ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29742024-11-19T01:47:33.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29752024-11-19T01:47:33.885ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29762024-11-19T01:47:33.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29772024-11-19T01:47:33.886ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29782024-11-19T01:47:33.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29792024-11-19T01:47:33.888ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29802024-11-19T01:47:33.891ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
29812024-11-19T01:47:33.891ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29822024-11-19T01:47:33.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29832024-11-19T01:47:33.900ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 6a0f8dba-9921-47cc-b46d-9464f538b1b5, kind: Service { id: 7dcf0d8a-40d8-45f8-a6f1-d545f08c056b }, name: Name("nexus-7dcf0d8a-40d8-45f8-a6f1-d545f08c056b"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9357d91c-cd60-4b75-8301-39bad393d1d3, kind: Service { id: 13c22fb6-5873-4940-9848-cc9130b7aa73 }, name: Name("nexus-13c22fb6-5873-4940-9848-cc9130b7aa73"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 90])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b12f392c-1b48-41cc-89b1-c909b6177044, kind: Service { id: d9867752-e046-43b5-9e2e-9abcfe9a206e }, name: Name("nexus-d9867752-e046-43b5-9e2e-9abcfe9a206e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 3ac0dcee-f159-48e6-bdef-a603be0719fd, kind: Service { id: a97a9e1e-ade8-4d48-bc64-1b5a63e68b91 }, name: Name("external-dns-a97a9e1e-ade8-4d48-bc64-1b5a63e68b91"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 223])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 830fafc3-9abd-43ea-95d4-d8bc4033ddcc, kind: Service { id: 52e4d19a-b928-4138-8817-4fd8deeaf29e }, name: Name("external-dns-52e4d19a-b928-4138-8817-4fd8deeaf29e"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
29842024-11-19T01:47:33.900ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29852024-11-19T01:47:33.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29862024-11-19T01:47:33.902ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29872024-11-19T01:47:33.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
29882024-11-19T01:47:33.903ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
29892024-11-19T01:47:33.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
29902024-11-19T01:47:33.915ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
29912024-11-19T01:47:33.915ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 3ac0dcee-f159-48e6-bdef-a603be0719fd, kind: Service { id: a97a9e1e-ade8-4d48-bc64-1b5a63e68b91 }, name: Name("external-dns-a97a9e1e-ade8-4d48-bc64-1b5a63e68b91"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 152, 223])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 830fafc3-9abd-43ea-95d4-d8bc4033ddcc, kind: Service { id: 52e4d19a-b928-4138-8817-4fd8deeaf29e }, name: Name("external-dns-52e4d19a-b928-4138-8817-4fd8deeaf29e"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 238])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 6a0f8dba-9921-47cc-b46d-9464f538b1b5, kind: Service { id: 7dcf0d8a-40d8-45f8-a6f1-d545f08c056b }, name: Name("nexus-7dcf0d8a-40d8-45f8-a6f1-d545f08c056b"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 211])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9357d91c-cd60-4b75-8301-39bad393d1d3, kind: Service { id: 13c22fb6-5873-4940-9848-cc9130b7aa73 }, name: Name("nexus-13c22fb6-5873-4940-9848-cc9130b7aa73"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 90])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b12f392c-1b48-41cc-89b1-c909b6177044, kind: Service { id: d9867752-e046-43b5-9e2e-9abcfe9a206e }, name: Name("nexus-d9867752-e046-43b5-9e2e-9abcfe9a206e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 169, 41])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
29922024-11-19T01:47:33.915ZDEBGnexus (service-firewall-rule-progator): resolved 2 rules for sleds background_task = service_firewall_rule_propagation
29932024-11-19T01:47:33.918ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29942024-11-19T01:47:33.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29952024-11-19T01:47:33.920ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
29962024-11-19T01:47:33.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29972024-11-19T01:47:33.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29982024-11-19T01:47:33.929ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
29992024-11-19T01:47:33.929ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7b6e10f-30a0-4dde-b2de-e15da5a349c5", "content-length": "134", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
30002024-11-19T01:47:33.929ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
30012024-11-19T01:47:33.929ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
30022024-11-19T01:47:33.929ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
30032024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
30042024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30052024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30062024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30072024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30082024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
30092024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
30102024-11-19T01:47:33.930ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
30112024-11-19T01:47:33.930ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30122024-11-19T01:47:33.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30132024-11-19T01:47:33.947ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30142024-11-19T01:47:33.949ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
30152024-11-19T01:47:33.949ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "984d401c-d239-4796-aca0-a521b9bbb08c", "content-length": "2", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
30162024-11-19T01:47:33.949ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
30172024-11-19T01:47:33.950ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fc2042e-ce7e-4ac3-bbfa-7106f467f806", "content-length": "51", "date": "Tue, 19 Nov 2024 01:47:33 GMT"} })
30182024-11-19T01:47:33.950ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
30192024-11-19T01:47:33.950ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
30202024-11-19T01:47:33.975ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 302245ee-8785-4baf-86ed-0823410907ec, time_created: 2024-11-19T01:32:30.165942Z, time_modified: 2024-11-19T01:32:30.165942Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(51474595840)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)) }]
30212024-11-19T01:47:33.975ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
30222024-11-19T01:47:33.975ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
30232024-11-19T01:47:33.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
30242024-11-19T01:47:33.977ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
30252024-11-19T01:47:33.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
30262024-11-19T01:47:33.979ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30272024-11-19T01:47:33.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
30282024-11-19T01:47:34.002ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f8c9c497-0c2c-4390-931e-7bcc9ea3436b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30292024-11-19T01:47:34.002ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f8c9c497-0c2c-4390-931e-7bcc9ea3436b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30302024-11-19T01:47:34.002ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c774a08-75b3-4d8e-b805-7642cc5d2419", "content-length": "691", "date": "Tue, 19 Nov 2024 01:47:34 GMT"} })
30312024-11-19T01:47:34.002ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
30322024-11-19T01:47:34.002ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
30332024-11-19T01:47:34.002ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
30342024-11-19T01:47:34.004ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2388 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f8c9c497-0c2c-4390-931e-7bcc9ea3436b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30352024-11-19T01:47:34.004ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 474.524456ms iteration = 32 status = {}
30362024-11-19T01:47:34.011ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
30372024-11-19T01:47:34.020ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "31932e37-d43e-4d83-adc3-3d1e4b902662", "date": "Tue, 19 Nov 2024 01:47:34 GMT"} })
30382024-11-19T01:47:34.021ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:496
30392024-11-19T01:47:34.021ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 491.104862ms
30402024-11-19T01:47:34.021ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 491.113773ms iteration = 4 status = {}
30412024-11-19T01:47:34.110ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "846fd155-7507-4873-b0a8-5d9ce255de93", "date": "Tue, 19 Nov 2024 01:47:34 GMT"} })
30422024-11-19T01:47:34.110ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
30432024-11-19T01:47:34.110ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
30442024-11-19T01:47:34.110ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2942dd1e-69a4-4b3e-aef3-d811e03543cd", "content-length": "682", "date": "Tue, 19 Nov 2024 01:47:34 GMT"} })
30452024-11-19T01:47:34.110ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30462024-11-19T01:47:34.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30472024-11-19T01:47:34.114ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30482024-11-19T01:47:34.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30492024-11-19T01:47:34.118ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30502024-11-19T01:47:34.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30512024-11-19T01:47:34.120ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30522024-11-19T01:47:34.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30532024-11-19T01:47:34.128ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 598.475072ms iteration = 31 status = {}
30542024-11-19T01:47:35.040ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 45bb9cb1-ce6b-493f-954b-4f5dac8bc7a2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30552024-11-19T01:47:35.040ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 45bb9cb1-ce6b-493f-954b-4f5dac8bc7a2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30562024-11-19T01:47:35.042ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2480 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 45bb9cb1-ce6b-493f-954b-4f5dac8bc7a2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30572024-11-19T01:47:36.062ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e8aac0af-f709-40da-9184-9fe57ba95893 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30582024-11-19T01:47:36.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e8aac0af-f709-40da-9184-9fe57ba95893 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30592024-11-19T01:47:36.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2738 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e8aac0af-f709-40da-9184-9fe57ba95893 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30602024-11-19T01:47:36.218ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 411f2f38-5bf7-4cac-a873-140ac8a9040b roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
30612024-11-19T01:47:36.218ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 411f2f38-5bf7-4cac-a873-140ac8a9040b resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
30622024-11-19T01:47:36.220ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2792 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 411f2f38-5bf7-4cac-a873-140ac8a9040b response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
30632024-11-19T01:47:37.085ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5afecc96-8723-49a1-aa6c-63c97c8d0dae roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30642024-11-19T01:47:37.085ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5afecc96-8723-49a1-aa6c-63c97c8d0dae resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30652024-11-19T01:47:37.088ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3037 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5afecc96-8723-49a1-aa6c-63c97c8d0dae response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30662024-11-19T01:47:38.111ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8e07a61c-71cd-4a18-8dc7-0176ad9c05f9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30672024-11-19T01:47:38.111ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8e07a61c-71cd-4a18-8dc7-0176ad9c05f9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30682024-11-19T01:47:38.113ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2342 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8e07a61c-71cd-4a18-8dc7-0176ad9c05f9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30692024-11-19T01:47:38.462ZDEBGnexus (producer-registration-task): registering / renewing oximeter producer lease with Nexus
30702024-11-19T01:47:38.462ZDEBGnexus (producer-registration-task): using nexus address for registration addr = [fd00:1122:3344:101::b]:12221
30712024-11-19T01:47:38.493ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
30722024-11-19T01:47:38.493ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:57794
30732024-11-19T01:47:38.493ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57794 req_id = 665c1c40-d9c8-4165-9282-e4782c2eb523 roles = RoleSet { roles: {} } uri = /metrics/producers
30742024-11-19T01:47:38.494ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57794 req_id = 665c1c40-d9c8-4165-9282-e4782c2eb523 resource = Database result = Ok(()) uri = /metrics/producers
30752024-11-19T01:47:38.534ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 13c22fb6-5873-4940-9848-cc9130b7aa73
30762024-11-19T01:47:38.534ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40765 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57794 req_id = 665c1c40-d9c8-4165-9282-e4782c2eb523 response_code = 201 uri = /metrics/producers
30772024-11-19T01:47:38.534ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "665c1c40-d9c8-4165-9282-e4782c2eb523", "content-length": "41", "date": "Tue, 19 Nov 2024 01:47:38 GMT"} })
30782024-11-19T01:47:38.536ZDEBGnexus (producer-registration-task): registered with nexus successfully
30792024-11-19T01:47:38.536ZDEBGnexus (producer-registration-task): pausing until time to renew lease lease_duration = 600s wait_period = 150s
30802024-11-19T01:47:39.136ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 82f51d19-464f-48f4-b78a-ac3debca25e6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30812024-11-19T01:47:39.136ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 82f51d19-464f-48f4-b78a-ac3debca25e6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30822024-11-19T01:47:39.138ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2424 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 82f51d19-464f-48f4-b78a-ac3debca25e6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30832024-11-19T01:47:40.160ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 48d765f4-9c58-4bb0-b037-1254f48f0114 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30842024-11-19T01:47:40.160ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 48d765f4-9c58-4bb0-b037-1254f48f0114 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30852024-11-19T01:47:40.163ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2856 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 48d765f4-9c58-4bb0-b037-1254f48f0114 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30862024-11-19T01:47:41.184ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ae8b304a-2c67-450b-a6dc-399f506d1130 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30872024-11-19T01:47:41.185ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ae8b304a-2c67-450b-a6dc-399f506d1130 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30882024-11-19T01:47:41.187ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2841 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ae8b304a-2c67-450b-a6dc-399f506d1130 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30892024-11-19T01:47:41.810ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1698 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 163c740a-45d4-4035-8be9-ef26ac6c72ae response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
30902024-11-19T01:47:42.211ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0731d825-66d3-4cd7-9f80-13be48ddccd7 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30912024-11-19T01:47:42.211ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0731d825-66d3-4cd7-9f80-13be48ddccd7 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30922024-11-19T01:47:42.213ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2086 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0731d825-66d3-4cd7-9f80-13be48ddccd7 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30932024-11-19T01:47:43.234ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 49fb5f0c-cfc7-404a-8e1a-f076cf30eeec roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30942024-11-19T01:47:43.234ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 49fb5f0c-cfc7-404a-8e1a-f076cf30eeec resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30952024-11-19T01:47:43.236ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2272 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 49fb5f0c-cfc7-404a-8e1a-f076cf30eeec response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
30962024-11-19T01:47:43.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 92 reason = Timeout
30972024-11-19T01:47:43.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30982024-11-19T01:47:43.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30992024-11-19T01:47:43.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 92 reason = Timeout
31002024-11-19T01:47:43.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31012024-11-19T01:47:43.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31022024-11-19T01:47:43.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31032024-11-19T01:47:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31042024-11-19T01:47:43.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31052024-11-19T01:47:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31062024-11-19T01:47:43.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31072024-11-19T01:47:43.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31082024-11-19T01:47:43.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31092024-11-19T01:47:43.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
31102024-11-19T01:47:43.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31112024-11-19T01:47:43.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31122024-11-19T01:47:43.555ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
31132024-11-19T01:47:43.555ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.058676ms iteration = 92 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
31142024-11-19T01:47:43.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31152024-11-19T01:47:43.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31162024-11-19T01:47:43.563ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 32.152884ms iteration = 92 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31172024-11-19T01:47:44.258ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9dc642b2-6937-4596-bc6c-949f13a1cfa5 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31182024-11-19T01:47:44.259ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9dc642b2-6937-4596-bc6c-949f13a1cfa5 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31192024-11-19T01:47:44.260ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1713 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9dc642b2-6937-4596-bc6c-949f13a1cfa5 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31202024-11-19T01:47:45.280ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b28b476d-7e39-4b23-9486-420f5c6f3df7 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31212024-11-19T01:47:45.280ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b28b476d-7e39-4b23-9486-420f5c6f3df7 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31222024-11-19T01:47:45.282ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2392 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b28b476d-7e39-4b23-9486-420f5c6f3df7 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31232024-11-19T01:47:46.283ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:49607
31242024-11-19T01:47:46.283ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49607 req_id = edb2bcdd-80ec-428b-a99d-90cf9048979f roles = RoleSet { roles: {} } uri = /metrics/producers
31252024-11-19T01:47:46.283ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49607 req_id = edb2bcdd-80ec-428b-a99d-90cf9048979f resource = Database result = Ok(()) uri = /metrics/producers
31262024-11-19T01:47:46.304ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 73802339-1d67-4783-a7cd-7cfb7c58f1bf roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31272024-11-19T01:47:46.304ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 73802339-1d67-4783-a7cd-7cfb7c58f1bf resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31282024-11-19T01:47:46.306ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1883 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 73802339-1d67-4783-a7cd-7cfb7c58f1bf response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31292024-11-19T01:47:46.319ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 1b898a1c-dd6e-4a5f-b7b4-a7823ab855c5
31302024-11-19T01:47:46.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36279 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:49607 req_id = edb2bcdd-80ec-428b-a99d-90cf9048979f response_code = 201 uri = /metrics/producers
31312024-11-19T01:47:47.327ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9b7b7591-4389-4cfc-a6f3-c08a4fee2879 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31322024-11-19T01:47:47.328ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9b7b7591-4389-4cfc-a6f3-c08a4fee2879 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31332024-11-19T01:47:47.330ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2358 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 9b7b7591-4389-4cfc-a6f3-c08a4fee2879 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31342024-11-19T01:47:48.354ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0318c4da-d828-4b16-9979-ffb15df0f0c2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31352024-11-19T01:47:48.354ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0318c4da-d828-4b16-9979-ffb15df0f0c2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31362024-11-19T01:47:48.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2499 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0318c4da-d828-4b16-9979-ffb15df0f0c2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31372024-11-19T01:47:49.378ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e90f92b1-84e7-44f5-9116-b8febd88e453 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31382024-11-19T01:47:49.379ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e90f92b1-84e7-44f5-9116-b8febd88e453 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31392024-11-19T01:47:49.381ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2349 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e90f92b1-84e7-44f5-9116-b8febd88e453 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31402024-11-19T01:47:50.404ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7c71c621-2abc-442b-b39f-ad2afc7fbd0b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31412024-11-19T01:47:50.404ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7c71c621-2abc-442b-b39f-ad2afc7fbd0b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31422024-11-19T01:47:50.406ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2121 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7c71c621-2abc-442b-b39f-ad2afc7fbd0b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31432024-11-19T01:47:51.428ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 72a64166-d687-45b3-a0e9-41a93823e6e9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31442024-11-19T01:47:51.428ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 72a64166-d687-45b3-a0e9-41a93823e6e9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31452024-11-19T01:47:51.430ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2274 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 72a64166-d687-45b3-a0e9-41a93823e6e9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31462024-11-19T01:47:52.452ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a320637b-f7c7-411f-a687-81f8b7333f34 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31472024-11-19T01:47:52.453ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a320637b-f7c7-411f-a687-81f8b7333f34 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31482024-11-19T01:47:52.455ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2636 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a320637b-f7c7-411f-a687-81f8b7333f34 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31492024-11-19T01:47:53.476ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f3d9929d-e1c4-47f3-97d0-ff784bcc3dab roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31502024-11-19T01:47:53.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f3d9929d-e1c4-47f3-97d0-ff784bcc3dab resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31512024-11-19T01:47:53.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2257 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f3d9929d-e1c4-47f3-97d0-ff784bcc3dab response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31522024-11-19T01:47:53.529ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 93 reason = Timeout
31532024-11-19T01:47:53.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31542024-11-19T01:47:53.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31552024-11-19T01:47:53.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31562024-11-19T01:47:53.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 93 reason = Timeout
31572024-11-19T01:47:53.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31582024-11-19T01:47:53.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31592024-11-19T01:47:53.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31602024-11-19T01:47:53.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31612024-11-19T01:47:53.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31622024-11-19T01:47:53.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31632024-11-19T01:47:53.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31642024-11-19T01:47:53.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31652024-11-19T01:47:53.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31662024-11-19T01:47:53.534ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.251818ms iteration = 93 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31672024-11-19T01:47:53.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31682024-11-19T01:47:53.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
31692024-11-19T01:47:53.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31702024-11-19T01:47:53.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31712024-11-19T01:47:53.549ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
31722024-11-19T01:47:53.549ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.608939ms iteration = 93 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
31732024-11-19T01:47:54.500ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 611d0ed8-f503-48d9-9a5e-f1459f29c25d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31742024-11-19T01:47:54.500ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 611d0ed8-f503-48d9-9a5e-f1459f29c25d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31752024-11-19T01:47:54.502ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2028 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 611d0ed8-f503-48d9-9a5e-f1459f29c25d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31762024-11-19T01:47:55.524ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 78be4b85-6125-44b7-9c3f-9b514b9c5a06 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31772024-11-19T01:47:55.524ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 78be4b85-6125-44b7-9c3f-9b514b9c5a06 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31782024-11-19T01:47:55.526ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2682 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 78be4b85-6125-44b7-9c3f-9b514b9c5a06 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31792024-11-19T01:47:56.548ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 20dcff88-daeb-4648-98cb-a8bd626f7a43 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31802024-11-19T01:47:56.549ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 20dcff88-daeb-4648-98cb-a8bd626f7a43 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31812024-11-19T01:47:56.551ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2612 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 20dcff88-daeb-4648-98cb-a8bd626f7a43 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31822024-11-19T01:47:56.811ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1633 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = af830b3d-ce07-4c08-9de4-4732a7d362c6 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
31832024-11-19T01:47:56.977ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:64740
31842024-11-19T01:47:56.977ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:64740 req_id = 9a2855bc-c738-4314-b56d-7fcafa45e33f roles = RoleSet { roles: {} } uri = /metrics/producers
31852024-11-19T01:47:56.977ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:64740 req_id = 9a2855bc-c738-4314-b56d-7fcafa45e33f resource = Database result = Ok(()) uri = /metrics/producers
31862024-11-19T01:47:57.019ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = e791ed89-1335-4e80-9045-8f1c5065489f
31872024-11-19T01:47:57.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42223 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:64740 req_id = 9a2855bc-c738-4314-b56d-7fcafa45e33f response_code = 201 uri = /metrics/producers
31882024-11-19T01:47:57.572ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 939a396c-0f6a-4570-ab6e-b24daceb472a roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31892024-11-19T01:47:57.573ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 939a396c-0f6a-4570-ab6e-b24daceb472a resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31902024-11-19T01:47:57.575ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2351 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 939a396c-0f6a-4570-ab6e-b24daceb472a response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31912024-11-19T01:47:58.596ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1624a6f2-f94a-47c3-8365-f2f05c954c40 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31922024-11-19T01:47:58.596ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1624a6f2-f94a-47c3-8365-f2f05c954c40 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31932024-11-19T01:47:58.598ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2141 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 1624a6f2-f94a-47c3-8365-f2f05c954c40 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31942024-11-19T01:47:59.620ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37049e6c-e027-4a8e-8714-871879566671 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31952024-11-19T01:47:59.621ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37049e6c-e027-4a8e-8714-871879566671 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31962024-11-19T01:47:59.623ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2452 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 37049e6c-e027-4a8e-8714-871879566671 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31972024-11-19T01:48:00.644ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aa4d459c-b3c6-4ed7-a967-b8e1ecc114a4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31982024-11-19T01:48:00.645ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aa4d459c-b3c6-4ed7-a967-b8e1ecc114a4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
31992024-11-19T01:48:00.646ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2171 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = aa4d459c-b3c6-4ed7-a967-b8e1ecc114a4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32002024-11-19T01:48:01.670ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 430182ee-f43e-45a7-b058-caba8b0169a4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32012024-11-19T01:48:01.671ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 430182ee-f43e-45a7-b058-caba8b0169a4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32022024-11-19T01:48:01.673ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2651 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 430182ee-f43e-45a7-b058-caba8b0169a4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32032024-11-19T01:48:02.695ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5a700c88-137e-47bd-bc81-83ea2fe736d2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32042024-11-19T01:48:02.695ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5a700c88-137e-47bd-bc81-83ea2fe736d2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32052024-11-19T01:48:02.697ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2233 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 5a700c88-137e-47bd-bc81-83ea2fe736d2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
32062024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 32 reason = Timeout
32072024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
32082024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 94 reason = Timeout
32092024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 32 reason = Timeout
32102024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 32 reason = Timeout
32112024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32122024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
32132024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 32 reason = Timeout
32142024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 32 reason = Timeout
32152024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 32 reason = Timeout
32162024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32172024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32182024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
32192024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
32202024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32212024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 33 reason = Timeout
32222024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32232024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
32242024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
32252024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32262024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
32272024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32282024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32292024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32302024-11-19T01:48:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32312024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 32 reason = Timeout
32322024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 94 reason = Timeout
32332024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
32342024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 32 reason = Timeout
32352024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
32362024-11-19T01:48:03.531ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
32372024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32382024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32392024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
32402024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32412024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
32422024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32432024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32442024-11-19T01:48:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32452024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 32 reason = Timeout
32462024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
32472024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 32 reason = Timeout
32482024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
32492024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 32 reason = Timeout
32502024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
32512024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 32 reason = Timeout
32522024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
32532024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
32542024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
32552024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
32562024-11-19T01:48:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
32572024-11-19T01:48:03.533ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.919872ms iteration = 32 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
32582024-11-19T01:48:03.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32592024-11-19T01:48:03.533ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 700932338 } })
32602024-11-19T01:48:03.533ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 701076350 } })
32612024-11-19T01:48:03.533ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
32622024-11-19T01:48:03.533ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
32632024-11-19T01:48:03.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32642024-11-19T01:48:03.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32652024-11-19T01:48:03.534ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32662024-11-19T01:48:03.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32672024-11-19T01:48:03.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32682024-11-19T01:48:03.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32692024-11-19T01:48:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32702024-11-19T01:48:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32712024-11-19T01:48:03.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32722024-11-19T01:48:03.537ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32732024-11-19T01:48:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32742024-11-19T01:48:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32752024-11-19T01:48:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
32762024-11-19T01:48:03.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32772024-11-19T01:48:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32782024-11-19T01:48:03.540ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32792024-11-19T01:48:03.540ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32802024-11-19T01:48:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32812024-11-19T01:48:03.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32822024-11-19T01:48:03.541ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 9.112692ms iteration = 32 status = {"errors":[],"records_set_to_done":[]}
32832024-11-19T01:48:03.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
32842024-11-19T01:48:03.542ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32852024-11-19T01:48:03.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
32862024-11-19T01:48:03.542ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
32872024-11-19T01:48:03.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32882024-11-19T01:48:03.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
32892024-11-19T01:48:03.545ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1933, tv_nsec: 713051755 } })
32902024-11-19T01:48:03.545ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
32912024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32922024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32932024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
32942024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32952024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32962024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32972024-11-19T01:48:03.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32982024-11-19T01:48:03.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32992024-11-19T01:48:03.548ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
33002024-11-19T01:48:03.548ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33012024-11-19T01:48:03.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33022024-11-19T01:48:03.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33032024-11-19T01:48:03.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
33042024-11-19T01:48:03.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33052024-11-19T01:48:03.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33062024-11-19T01:48:03.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33072024-11-19T01:48:03.552ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33082024-11-19T01:48:03.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33092024-11-19T01:48:03.568ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
33102024-11-19T01:48:03.568ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 38.389466ms iteration = 94 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
33112024-11-19T01:48:03.572ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33122024-11-19T01:48:03.572ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
33132024-11-19T01:48:03.572ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33142024-11-19T01:48:03.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33152024-11-19T01:48:03.574ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33162024-11-19T01:48:03.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
33172024-11-19T01:48:03.578ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 45.671287ms iteration = 32 status = {"errors":[],"garbage_collect_requested":[]}
33182024-11-19T01:48:03.578ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33192024-11-19T01:48:03.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33202024-11-19T01:48:03.581ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33212024-11-19T01:48:03.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33222024-11-19T01:48:03.582ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33232024-11-19T01:48:03.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33242024-11-19T01:48:03.585ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33252024-11-19T01:48:03.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33262024-11-19T01:48:03.586ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
33272024-11-19T01:48:03.586ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 55.222508ms iteration = 32 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
33282024-11-19T01:48:03.587ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33292024-11-19T01:48:03.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33302024-11-19T01:48:03.589ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.783543ms iteration = 94 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33312024-11-19T01:48:03.606ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33322024-11-19T01:48:03.606ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
33332024-11-19T01:48:03.606ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33342024-11-19T01:48:03.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
33352024-11-19T01:48:03.615ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.761663ms iteration = 32 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
33362024-11-19T01:48:03.616ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 84.630993ms iteration = 32 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
33372024-11-19T01:48:03.617ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
33382024-11-19T01:48:03.617ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33392024-11-19T01:48:03.617ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e7e7a09-83cc-44ca-aa72-3fc4139807b9", "content-length": "26", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
33402024-11-19T01:48:03.617ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33412024-11-19T01:48:03.619ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33422024-11-19T01:48:03.619ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33432024-11-19T01:48:03.622ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
33442024-11-19T01:48:03.622ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33452024-11-19T01:48:03.622ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "68f61585-b6aa-4c18-a537-d5535f4f8a7c", "content-length": "26", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
33462024-11-19T01:48:03.622ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33472024-11-19T01:48:03.624ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
33482024-11-19T01:48:03.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
33492024-11-19T01:48:03.624ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.119487ms iteration = 32 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
33502024-11-19T01:48:03.624ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33512024-11-19T01:48:03.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33522024-11-19T01:48:03.626ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.394244ms iteration = 32 status = 0
33532024-11-19T01:48:03.631ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
33542024-11-19T01:48:03.631ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33552024-11-19T01:48:03.632ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f4f39a5-c3ed-41c7-ac04-3cf32a9e37bb", "content-length": "26", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
33562024-11-19T01:48:03.632ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33572024-11-19T01:48:03.634ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
33582024-11-19T01:48:03.645ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
33592024-11-19T01:48:03.645ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
33602024-11-19T01:48:03.645ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 114.323164ms iteration = 32 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
33612024-11-19T01:48:03.650ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33622024-11-19T01:48:03.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33632024-11-19T01:48:03.664ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 133.968396ms iteration = 36 status = {"physical_disks_added":0}
33642024-11-19T01:48:03.701ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 171.550658ms iteration = 32 status = {}
33652024-11-19T01:48:03.716ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
33662024-11-19T01:48:03.716ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33672024-11-19T01:48:03.722ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
33682024-11-19T01:48:03.723ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c5e42b5-89ce-4394-8dd1-d1b571ca3725", "content-length": "2", "date": "Tue, 19 Nov 2024 01:48:02 GMT"} })
33692024-11-19T01:48:03.724ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
33702024-11-19T01:48:03.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
33712024-11-19T01:48:03.726ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 196.87585ms iteration = 32 status = null
33722024-11-19T01:48:03.737ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33732024-11-19T01:48:03.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33742024-11-19T01:48:03.739ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33752024-11-19T01:48:03.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33762024-11-19T01:48:03.741ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33772024-11-19T01:48:03.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33782024-11-19T01:48:03.742ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33792024-11-19T01:48:03.744ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
33802024-11-19T01:48:03.744ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
33812024-11-19T01:48:03.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
33822024-11-19T01:48:03.803ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
33832024-11-19T01:48:03.804ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33842024-11-19T01:48:03.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33852024-11-19T01:48:03.806ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
33862024-11-19T01:48:03.806ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33872024-11-19T01:48:03.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33882024-11-19T01:48:03.817ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
33892024-11-19T01:48:03.817ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "efbc8c2a-3947-431f-aaab-e4de0674a825", "content-length": "134", "date": "Tue, 19 Nov 2024 01:48:02 GMT"} })
33902024-11-19T01:48:03.817ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
33912024-11-19T01:48:03.817ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
33922024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
33932024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
33942024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33952024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33962024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33972024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
33982024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
33992024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
34002024-11-19T01:48:03.818ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
34012024-11-19T01:48:03.818ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34022024-11-19T01:48:03.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34032024-11-19T01:48:03.820ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
34042024-11-19T01:48:03.820ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f65d997-dd51-47a0-88c7-7e625b9a794e", "content-length": "2", "date": "Tue, 19 Nov 2024 01:48:02 GMT"} })
34052024-11-19T01:48:03.820ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
34062024-11-19T01:48:03.821ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f8c6c4f-3d2a-484f-8671-02dcc0ace728", "content-length": "51", "date": "Tue, 19 Nov 2024 01:48:02 GMT"} })
34072024-11-19T01:48:03.821ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
34082024-11-19T01:48:03.821ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34092024-11-19T01:48:03.824ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 381bf569-4a17-426b-b9ed-3866a2935afc roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34102024-11-19T01:48:03.824ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 381bf569-4a17-426b-b9ed-3866a2935afc resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34112024-11-19T01:48:03.824ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0a5e863-47e1-4b16-9c02-4638aefd2f68", "content-length": "372", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
34122024-11-19T01:48:03.824ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "44e373d8-1579-4951-9534-34ed59ed3dac", "content-length": "691", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
34132024-11-19T01:48:03.824ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34142024-11-19T01:48:03.824ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
34152024-11-19T01:48:03.824ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
34162024-11-19T01:48:03.824ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
34172024-11-19T01:48:03.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34182024-11-19T01:48:03.826ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2429 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 381bf569-4a17-426b-b9ed-3866a2935afc response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34192024-11-19T01:48:03.826ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 296.546942ms iteration = 33 status = {}
34202024-11-19T01:48:03.864ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
34212024-11-19T01:48:03.864ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
34222024-11-19T01:48:03.864ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ed527413-2578-446e-8ca4-6b70bd48ac7f", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
34232024-11-19T01:48:03.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34242024-11-19T01:48:03.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34252024-11-19T01:48:03.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34262024-11-19T01:48:03.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34272024-11-19T01:48:03.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34282024-11-19T01:48:03.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34292024-11-19T01:48:03.874ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
34302024-11-19T01:48:03.874ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
34312024-11-19T01:48:03.877ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3016e415-c6ee-4775-93f5-b4b773293853", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
34322024-11-19T01:48:03.879ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 347.8639ms iteration = 34 status = {}
34332024-11-19T01:48:03.931ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7f7c4429-5dd3-4f17-8d63-184f89668db4", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
34342024-11-19T01:48:03.931ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
34352024-11-19T01:48:03.931ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34362024-11-19T01:48:03.931ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "92ad0fd7-2e65-457f-800c-a37f7033bd2a", "content-length": "682", "date": "Tue, 19 Nov 2024 01:48:03 GMT"} })
34372024-11-19T01:48:03.931ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34382024-11-19T01:48:03.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34392024-11-19T01:48:03.935ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34402024-11-19T01:48:03.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34412024-11-19T01:48:03.938ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34422024-11-19T01:48:03.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34432024-11-19T01:48:03.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34442024-11-19T01:48:03.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34452024-11-19T01:48:03.948ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 418.107447ms iteration = 32 status = {}
34462024-11-19T01:48:04.854ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f9fa1a98-5853-4e69-96ed-42f30e500580 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34472024-11-19T01:48:04.854ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f9fa1a98-5853-4e69-96ed-42f30e500580 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34482024-11-19T01:48:04.857ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2630 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f9fa1a98-5853-4e69-96ed-42f30e500580 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34492024-11-19T01:48:05.878ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 504ba560-b39c-4ac5-940c-236eff57636e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34502024-11-19T01:48:05.879ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 504ba560-b39c-4ac5-940c-236eff57636e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34512024-11-19T01:48:05.881ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2556 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 504ba560-b39c-4ac5-940c-236eff57636e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34522024-11-19T01:48:06.252ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 94529536-12bd-4fc5-b68b-0efbb16e6eb7 roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
34532024-11-19T01:48:06.252ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 94529536-12bd-4fc5-b68b-0efbb16e6eb7 resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
34542024-11-19T01:48:06.255ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3442 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 94529536-12bd-4fc5-b68b-0efbb16e6eb7 response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
34552024-11-19T01:48:06.902ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f9361777-cca1-4b65-a64a-e79f0f5fa969 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34562024-11-19T01:48:06.903ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f9361777-cca1-4b65-a64a-e79f0f5fa969 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34572024-11-19T01:48:06.904ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2319 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f9361777-cca1-4b65-a64a-e79f0f5fa969 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34582024-11-19T01:48:07.925ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 81285e50-9eb5-43b2-8cf1-219e8c185e5f roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34592024-11-19T01:48:07.926ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 81285e50-9eb5-43b2-8cf1-219e8c185e5f resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34602024-11-19T01:48:07.929ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3417 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 81285e50-9eb5-43b2-8cf1-219e8c185e5f response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34612024-11-19T01:48:08.950ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 872c1c24-0f21-4423-8508-4303302759b1 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34622024-11-19T01:48:08.951ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 872c1c24-0f21-4423-8508-4303302759b1 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34632024-11-19T01:48:08.953ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2476 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 872c1c24-0f21-4423-8508-4303302759b1 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34642024-11-19T01:48:09.974ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a826e4ee-5071-413f-b235-6ca9a7dccfdb roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34652024-11-19T01:48:09.974ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a826e4ee-5071-413f-b235-6ca9a7dccfdb resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34662024-11-19T01:48:09.976ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2182 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a826e4ee-5071-413f-b235-6ca9a7dccfdb response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34672024-11-19T01:48:10.996ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b9f3f6b8-1819-42b5-914f-d620758dc1e6 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34682024-11-19T01:48:10.997ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b9f3f6b8-1819-42b5-914f-d620758dc1e6 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34692024-11-19T01:48:10.999ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2585 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b9f3f6b8-1819-42b5-914f-d620758dc1e6 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34702024-11-19T01:48:11.423ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::1]:33060
34712024-11-19T01:48:11.423ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33060 req_id = e48724b4-050c-4874-a364-439d03800d62 roles = RoleSet { roles: {} } uri = /metrics/producers
34722024-11-19T01:48:11.423ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33060 req_id = e48724b4-050c-4874-a364-439d03800d62 resource = Database result = Ok(()) uri = /metrics/producers
34732024-11-19T01:48:11.463ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 302245ee-8785-4baf-86ed-0823410907ec
34742024-11-19T01:48:11.463ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40165 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::1]:33060 req_id = e48724b4-050c-4874-a364-439d03800d62 response_code = 201 uri = /metrics/producers
34752024-11-19T01:48:11.809ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1298 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 084dfa81-78f6-45f3-9aa6-32873f0b8a72 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
34762024-11-19T01:48:12.024ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3256ed26-672f-4edb-8ed4-b0fcd25fa154 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34772024-11-19T01:48:12.024ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3256ed26-672f-4edb-8ed4-b0fcd25fa154 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34782024-11-19T01:48:12.026ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2223 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3256ed26-672f-4edb-8ed4-b0fcd25fa154 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34792024-11-19T01:48:13.048ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 574d8155-0030-4372-98c0-f607b3451bfd roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34802024-11-19T01:48:13.049ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 574d8155-0030-4372-98c0-f607b3451bfd resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34812024-11-19T01:48:13.051ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2446 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 574d8155-0030-4372-98c0-f607b3451bfd response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
34822024-11-19T01:48:13.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 95 reason = Timeout
34832024-11-19T01:48:13.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34842024-11-19T01:48:13.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34852024-11-19T01:48:13.531ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 95 reason = Timeout
34862024-11-19T01:48:13.531ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34872024-11-19T01:48:13.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34882024-11-19T01:48:13.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34892024-11-19T01:48:13.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34902024-11-19T01:48:13.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34912024-11-19T01:48:13.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34922024-11-19T01:48:13.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34932024-11-19T01:48:13.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34942024-11-19T01:48:13.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34952024-11-19T01:48:13.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34962024-11-19T01:48:13.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34972024-11-19T01:48:13.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
34982024-11-19T01:48:13.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34992024-11-19T01:48:13.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35002024-11-19T01:48:13.550ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
35012024-11-19T01:48:13.550ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.485007ms iteration = 95 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
35022024-11-19T01:48:13.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.205038ms iteration = 95 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35032024-11-19T01:48:14.072ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c6f09fcc-b664-48e3-97ea-f789ce686f6d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35042024-11-19T01:48:14.073ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c6f09fcc-b664-48e3-97ea-f789ce686f6d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35052024-11-19T01:48:14.075ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2469 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c6f09fcc-b664-48e3-97ea-f789ce686f6d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35062024-11-19T01:48:15.097ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = dce9682f-bad7-44fc-b036-8d8f2e310715 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35072024-11-19T01:48:15.098ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = dce9682f-bad7-44fc-b036-8d8f2e310715 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35082024-11-19T01:48:15.099ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2307 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = dce9682f-bad7-44fc-b036-8d8f2e310715 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35092024-11-19T01:48:16.122ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a65809d8-1df8-48df-abda-a060a30c505c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35102024-11-19T01:48:16.122ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a65809d8-1df8-48df-abda-a060a30c505c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35112024-11-19T01:48:16.124ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2445 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a65809d8-1df8-48df-abda-a060a30c505c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35122024-11-19T01:48:17.146ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a28bf18b-5a9e-4796-9f9e-eaa636380990 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35132024-11-19T01:48:17.146ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a28bf18b-5a9e-4796-9f9e-eaa636380990 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35142024-11-19T01:48:17.148ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2224 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a28bf18b-5a9e-4796-9f9e-eaa636380990 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35152024-11-19T01:48:17.797ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:47792
35162024-11-19T01:48:17.797ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:47792 req_id = e24d9a66-1b61-425b-8568-123ef907b1ec roles = RoleSet { roles: {} } uri = /metrics/producers
35172024-11-19T01:48:17.797ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:47792 req_id = e24d9a66-1b61-425b-8568-123ef907b1ec resource = Database result = Ok(()) uri = /metrics/producers
35182024-11-19T01:48:17.836ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 128022c3-c3e6-460c-994e-e9c9cf944010
35192024-11-19T01:48:17.836ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38861 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:47792 req_id = e24d9a66-1b61-425b-8568-123ef907b1ec response_code = 201 uri = /metrics/producers
35202024-11-19T01:48:18.169ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 67ef731e-4c3e-47cf-aafe-c0e4f5b850e5 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35212024-11-19T01:48:18.170ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 67ef731e-4c3e-47cf-aafe-c0e4f5b850e5 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35222024-11-19T01:48:18.171ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2205 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 67ef731e-4c3e-47cf-aafe-c0e4f5b850e5 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35232024-11-19T01:48:19.196ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c5f63559-c754-47b5-a3ba-274aa1055a8c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35242024-11-19T01:48:19.196ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c5f63559-c754-47b5-a3ba-274aa1055a8c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35252024-11-19T01:48:19.198ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2255 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = c5f63559-c754-47b5-a3ba-274aa1055a8c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35262024-11-19T01:48:20.219ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 25ad845a-02b8-435f-86c1-374c9e03e50e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35272024-11-19T01:48:20.220ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 25ad845a-02b8-435f-86c1-374c9e03e50e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35282024-11-19T01:48:20.222ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2302 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 25ad845a-02b8-435f-86c1-374c9e03e50e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35292024-11-19T01:48:21.244ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8d95dc91-cce7-4161-81e0-8f97f6b2b916 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35302024-11-19T01:48:21.244ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8d95dc91-cce7-4161-81e0-8f97f6b2b916 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35312024-11-19T01:48:21.246ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2312 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 8d95dc91-cce7-4161-81e0-8f97f6b2b916 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35322024-11-19T01:48:22.269ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2945a6aa-6ab6-4cea-b353-4ac044a1fba4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35332024-11-19T01:48:22.269ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2945a6aa-6ab6-4cea-b353-4ac044a1fba4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35342024-11-19T01:48:22.271ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2522 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2945a6aa-6ab6-4cea-b353-4ac044a1fba4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35352024-11-19T01:48:23.293ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6fba82f2-3b43-480c-bbe2-8505b87275d0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35362024-11-19T01:48:23.294ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6fba82f2-3b43-480c-bbe2-8505b87275d0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35372024-11-19T01:48:23.296ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2532 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6fba82f2-3b43-480c-bbe2-8505b87275d0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35382024-11-19T01:48:23.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 96 reason = Timeout
35392024-11-19T01:48:23.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35402024-11-19T01:48:23.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35412024-11-19T01:48:23.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35422024-11-19T01:48:23.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 96 reason = Timeout
35432024-11-19T01:48:23.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35442024-11-19T01:48:23.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35452024-11-19T01:48:23.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35462024-11-19T01:48:23.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35472024-11-19T01:48:23.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35482024-11-19T01:48:23.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35492024-11-19T01:48:23.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35502024-11-19T01:48:23.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35512024-11-19T01:48:23.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
35522024-11-19T01:48:23.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35532024-11-19T01:48:23.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35542024-11-19T01:48:23.550ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
35552024-11-19T01:48:23.550ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 20.483987ms iteration = 96 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
35562024-11-19T01:48:23.567ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35572024-11-19T01:48:23.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35582024-11-19T01:48:23.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.092594ms iteration = 96 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35592024-11-19T01:48:24.317ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0486a199-2f9b-4ac9-9191-8bfb8005ab07 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35602024-11-19T01:48:24.318ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0486a199-2f9b-4ac9-9191-8bfb8005ab07 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35612024-11-19T01:48:24.319ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1974 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0486a199-2f9b-4ac9-9191-8bfb8005ab07 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35622024-11-19T01:48:25.342ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2cf9adf4-5b75-4324-836c-f316b5d60288 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35632024-11-19T01:48:25.343ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2cf9adf4-5b75-4324-836c-f316b5d60288 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35642024-11-19T01:48:25.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2773 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2cf9adf4-5b75-4324-836c-f316b5d60288 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35652024-11-19T01:48:26.367ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7cd09ccc-c333-4da3-bec3-d13f043ab4fd roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35662024-11-19T01:48:26.368ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7cd09ccc-c333-4da3-bec3-d13f043ab4fd resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35672024-11-19T01:48:26.369ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2268 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 7cd09ccc-c333-4da3-bec3-d13f043ab4fd response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35682024-11-19T01:48:26.809ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1393 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 8505b34f-0d9f-4c73-a51b-5a28ef8dc865 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
35692024-11-19T01:48:27.392ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 430d6efe-c34e-4bf6-a119-75e13b1a9e5c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35702024-11-19T01:48:27.392ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 430d6efe-c34e-4bf6-a119-75e13b1a9e5c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35712024-11-19T01:48:27.394ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2489 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 430d6efe-c34e-4bf6-a119-75e13b1a9e5c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35722024-11-19T01:48:28.416ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 19e00890-1221-4aaa-85d5-815e3054183c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35732024-11-19T01:48:28.416ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 19e00890-1221-4aaa-85d5-815e3054183c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35742024-11-19T01:48:28.418ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1990 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 19e00890-1221-4aaa-85d5-815e3054183c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35752024-11-19T01:48:28.638ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:64696
35762024-11-19T01:48:28.638ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64696 req_id = f511dc78-81a1-46bc-a949-2869a3a8730b roles = RoleSet { roles: {} } uri = /metrics/producers
35772024-11-19T01:48:28.639ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64696 req_id = f511dc78-81a1-46bc-a949-2869a3a8730b resource = Database result = Ok(()) uri = /metrics/producers
35782024-11-19T01:48:28.675ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = 60df7fc2-269e-4e73-a446-d55cd69c6f17
35792024-11-19T01:48:28.675ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37077 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:64696 req_id = f511dc78-81a1-46bc-a949-2869a3a8730b response_code = 201 uri = /metrics/producers
35802024-11-19T01:48:29.440ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = edfbe523-bf08-44d8-91ad-83d28dfbb679 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35812024-11-19T01:48:29.440ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = edfbe523-bf08-44d8-91ad-83d28dfbb679 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35822024-11-19T01:48:29.442ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2527 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = edfbe523-bf08-44d8-91ad-83d28dfbb679 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35832024-11-19T01:48:30.463ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d899b06d-442e-4c75-9004-03c865a7aaec roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35842024-11-19T01:48:30.463ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d899b06d-442e-4c75-9004-03c865a7aaec resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35852024-11-19T01:48:30.465ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2393 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = d899b06d-442e-4c75-9004-03c865a7aaec response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35862024-11-19T01:48:31.489ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0b045d26-fa90-45bb-86ad-bb989c99beaf roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35872024-11-19T01:48:31.489ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0b045d26-fa90-45bb-86ad-bb989c99beaf resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35882024-11-19T01:48:31.491ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2576 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0b045d26-fa90-45bb-86ad-bb989c99beaf response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35892024-11-19T01:48:32.514ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 17b4dccb-5052-498b-901b-bd0c6afd230e roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35902024-11-19T01:48:32.515ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 17b4dccb-5052-498b-901b-bd0c6afd230e resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35912024-11-19T01:48:32.516ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1970 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 17b4dccb-5052-498b-901b-bd0c6afd230e response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
35922024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 17 reason = Timeout
35932024-11-19T01:48:33.529ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2024-11-19 01:38:33.529799123 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
35942024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
35952024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 18 reason = Timeout
35962024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 33 reason = Timeout
35972024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 18 reason = Timeout
35982024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 18 reason = Timeout
35992024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36002024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
36012024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36022024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 19 reason = Timeout
36032024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 18 reason = Timeout
36042024-11-19T01:48:33.529ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
36052024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 33 reason = Timeout
36062024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36072024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
36082024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 18 reason = Timeout
36092024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 19 reason = Timeout
36102024-11-19T01:48:33.530ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:83 reason = blueprint disabled target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120
36112024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 11.63\u{b5}s iteration = 18 status = {"enabled":false,"target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120"}
36122024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 33 reason = Timeout
36132024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36142024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 97 reason = Timeout
36152024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 18 reason = Timeout
36162024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36172024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36182024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36192024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
36202024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
36212024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
36222024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
36232024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36242024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 33 reason = Timeout
36252024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
36262024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Timeout
36272024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36282024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 34 reason = Timeout
36292024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
36302024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 33 reason = Timeout
36312024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36322024-11-19T01:48:33.530ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1e98d393-d940-4def-ba7c-d8c67ea23aa1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("e48e23d9-6a0a-4bc6-9e0b-f83ba78952f1.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("efd7fee1-3fb4-451f-a329-22d734d942b3.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:1::1))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 697629446 } })
36332024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 17 reason = Timeout
36342024-11-19T01:48:33.530ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("52e4d19a-b928-4138-8817-4fd8deeaf29e.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("a97a9e1e-ade8-4d48-bc64-1b5a63e68b91.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::8))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 697638436 } })
36352024-11-19T01:48:33.530ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:56
36362024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 33 reason = Timeout
36372024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
36382024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36392024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
36402024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
36412024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36422024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36432024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
36442024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36452024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
36462024-11-19T01:48:33.530ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.10692ms iteration = 18 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
36472024-11-19T01:48:33.531ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
36482024-11-19T01:48:33.531ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.12039ms iteration = 18 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
36492024-11-19T01:48:33.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36502024-11-19T01:48:33.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36512024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Timeout
36522024-11-19T01:48:33.532ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
36532024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 33 reason = Timeout
36542024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36552024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
36562024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 97 reason = Timeout
36572024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36582024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 33 reason = Timeout
36592024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
36602024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
36612024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 17 reason = Timeout
36622024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
36632024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36642024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
36652024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
36662024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36672024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36682024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
36692024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36702024-11-19T01:48:33.532ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2024-11-19 01:38:33.529799123 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
36712024-11-19T01:48:33.532ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.981714ms iteration = 17 status = {"expiration":"2024-11-19T01:38:33.529799123Z","pruned":[]}
36722024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 33 reason = Timeout
36732024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 17 reason = Timeout
36742024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
36752024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
36762024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 33 reason = Timeout
36772024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 33 reason = Timeout
36782024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
36792024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
36802024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 33 reason = Timeout
36812024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
36822024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 17 reason = Timeout
36832024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
36842024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
36852024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
36862024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.690406ms iteration = 33 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36872024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
36882024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
36892024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
36902024-11-19T01:48:33.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
36912024-11-19T01:48:33.534ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 701349282 } })
36922024-11-19T01:48:33.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36932024-11-19T01:48:33.534ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1131
36942024-11-19T01:48:33.535ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 702439492 } })
36952024-11-19T01:48:33.535ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1131
36962024-11-19T01:48:33.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36972024-11-19T01:48:33.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
36982024-11-19T01:48:33.536ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36992024-11-19T01:48:33.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37002024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37012024-11-19T01:48:33.538ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 00b6f4dc-4ec2-43ec-bfa8-b87d02d9d600 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
37022024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37032024-11-19T01:48:33.538ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 00b6f4dc-4ec2-43ec-bfa8-b87d02d9d600 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
37042024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37052024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37062024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37072024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37082024-11-19T01:48:33.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
37092024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37102024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37112024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
37122024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37132024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37142024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37152024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37162024-11-19T01:48:33.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37172024-11-19T01:48:33.542ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37182024-11-19T01:48:33.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
37192024-11-19T01:48:33.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
37202024-11-19T01:48:33.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37212024-11-19T01:48:33.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
37222024-11-19T01:48:33.546ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37232024-11-19T01:48:33.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
37242024-11-19T01:48:33.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37252024-11-19T01:48:33.548ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37262024-11-19T01:48:33.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
37272024-11-19T01:48:33.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37282024-11-19T01:48:33.551ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37292024-11-19T01:48:33.551ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37302024-11-19T01:48:33.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37312024-11-19T01:48:33.553ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 20.721231ms iteration = 17 status = {"errors":[],"found_port_ok":[]}
37322024-11-19T01:48:33.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 18629 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 00b6f4dc-4ec2-43ec-bfa8-b87d02d9d600 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
37332024-11-19T01:48:33.556ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37342024-11-19T01:48:33.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37352024-11-19T01:48:33.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37362024-11-19T01:48:33.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37372024-11-19T01:48:33.557ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37382024-11-19T01:48:33.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37392024-11-19T01:48:33.558ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37402024-11-19T01:48:33.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37412024-11-19T01:48:33.558ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-302245ee-8785-4baf-86ed-0823410907ec.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(fd00:1122:3344:101::2))) }], valid_until: Instant { tv_sec: 1963, tv_nsec: 725756109 } })
37422024-11-19T01:48:33.558ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1131 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
37432024-11-19T01:48:33.592ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37442024-11-19T01:48:33.594ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37452024-11-19T01:48:33.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37462024-11-19T01:48:33.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
37472024-11-19T01:48:33.595ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
37482024-11-19T01:48:33.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
37492024-11-19T01:48:33.596ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
37502024-11-19T01:48:33.596ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
37512024-11-19T01:48:33.596ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37522024-11-19T01:48:33.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
37532024-11-19T01:48:33.597ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37542024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
37552024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
37562024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
37572024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37582024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 67.891594ms iteration = 33 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
37592024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37602024-11-19T01:48:33.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37612024-11-19T01:48:33.599ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
37622024-11-19T01:48:33.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
37632024-11-19T01:48:33.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37642024-11-19T01:48:33.601ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37652024-11-19T01:48:33.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37662024-11-19T01:48:33.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
37672024-11-19T01:48:33.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37682024-11-19T01:48:33.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
37692024-11-19T01:48:33.602ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37702024-11-19T01:48:33.603ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
37712024-11-19T01:48:33.603ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:171 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
37722024-11-19T01:48:33.603ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 70.020152ms iteration = 17 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
37732024-11-19T01:48:33.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
37742024-11-19T01:48:33.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37752024-11-19T01:48:33.610ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.916526ms iteration = 33 status = {"errors":[],"garbage_collect_requested":[]}
37762024-11-19T01:48:33.611ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37772024-11-19T01:48:33.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
37782024-11-19T01:48:33.613ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 79.813997ms iteration = 33 status = {"errors":[],"records_set_to_done":[]}
37792024-11-19T01:48:33.613ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37802024-11-19T01:48:33.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
37812024-11-19T01:48:33.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37822024-11-19T01:48:33.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37832024-11-19T01:48:33.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37842024-11-19T01:48:33.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
37852024-11-19T01:48:33.617ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37862024-11-19T01:48:33.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37872024-11-19T01:48:33.618ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:555 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
37882024-11-19T01:48:33.618ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 85.970007ms iteration = 33 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
37892024-11-19T01:48:33.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
37902024-11-19T01:48:33.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37912024-11-19T01:48:33.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37922024-11-19T01:48:33.628ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37932024-11-19T01:48:33.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37942024-11-19T01:48:33.633ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37952024-11-19T01:48:33.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37962024-11-19T01:48:33.635ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
37972024-11-19T01:48:33.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
37982024-11-19T01:48:33.638ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
37992024-11-19T01:48:33.638ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 108.185625ms iteration = 97 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
38002024-11-19T01:48:33.645ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
38012024-11-19T01:48:33.645ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
38022024-11-19T01:48:33.645ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
38032024-11-19T01:48:33.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38042024-11-19T01:48:33.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38052024-11-19T01:48:33.648ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
38062024-11-19T01:48:33.648ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 0 version = 2
38072024-11-19T01:48:33.648ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38082024-11-19T01:48:33.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38092024-11-19T01:48:33.649ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 116.094716ms iteration = 33 status = {"errors":[],"requests_created_ok":[],"start_invoked_ok":[]}
38102024-11-19T01:48:33.650ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38112024-11-19T01:48:33.650ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
38122024-11-19T01:48:33.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
38132024-11-19T01:48:33.650ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
38142024-11-19T01:48:33.650ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:140 found = 0 not_ready_to_be_deleted = 0
38152024-11-19T01:48:33.650ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 119.764972ms iteration = 17 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
38162024-11-19T01:48:33.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38172024-11-19T01:48:33.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
38182024-11-19T01:48:33.651ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 119.249593ms iteration = 97 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38192024-11-19T01:48:33.652ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
38202024-11-19T01:48:33.652ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
38212024-11-19T01:48:33.652ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 120.147938ms iteration = 33 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
38222024-11-19T01:48:33.654ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38232024-11-19T01:48:33.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38242024-11-19T01:48:33.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38252024-11-19T01:48:33.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38262024-11-19T01:48:33.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38272024-11-19T01:48:33.655ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38282024-11-19T01:48:33.655ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38292024-11-19T01:48:33.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38302024-11-19T01:48:33.656ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 11d9bf96-5d56-48be-8c8a-411f4aeae63f dns_zone_name = oxide.test found_so_far = 1 version = 2
38312024-11-19T01:48:33.656ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
38322024-11-19T01:48:33.656ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2024-11-19 01:32:36.054328 UTC dns_group = external generation = 2
38332024-11-19T01:48:33.656ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 126.90757ms iteration = 18 status = {"generation":2}
38342024-11-19T01:48:33.657ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
38352024-11-19T01:48:33.657ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38362024-11-19T01:48:33.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38372024-11-19T01:48:33.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
38382024-11-19T01:48:33.667ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
38392024-11-19T01:48:33.669ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38402024-11-19T01:48:33.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 302245ee-8785-4baf-86ed-0823410907ec, lookup_type: ById(302245ee-8785-4baf-86ed-0823410907ec) } result = Ok(())
38412024-11-19T01:48:33.671ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38422024-11-19T01:48:33.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38432024-11-19T01:48:33.674ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 143.529927ms iteration = 33 status = 0
38442024-11-19T01:48:33.679ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
38452024-11-19T01:48:33.679ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38462024-11-19T01:48:33.680ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5351b8d4-927e-44a9-9e9d-42ed29414187", "content-length": "26", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
38472024-11-19T01:48:33.680ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38482024-11-19T01:48:33.681ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
38492024-11-19T01:48:33.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38502024-11-19T01:48:33.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38512024-11-19T01:48:33.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38522024-11-19T01:48:33.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38532024-11-19T01:48:33.687ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1139 zone_address = fd00:1122:3344:101::2
38542024-11-19T01:48:33.687ZDEBGnexus (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38552024-11-19T01:48:33.687ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38562024-11-19T01:48:33.688ZDEBGnexus (MgsClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "404c20a7-39b8-49ba-9934-f5fba298dd4d", "content-length": "26", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
38572024-11-19T01:48:33.688ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1144 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38582024-11-19T01:48:33.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38592024-11-19T01:48:33.688ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 155.121654ms iteration = 33 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[]}
38602024-11-19T01:48:33.689ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38612024-11-19T01:48:33.690ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_v4_garbage_collector file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
38622024-11-19T01:48:33.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38632024-11-19T01:48:33.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38642024-11-19T01:48:33.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38652024-11-19T01:48:33.694ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38662024-11-19T01:48:33.695ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
38672024-11-19T01:48:33.695ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38682024-11-19T01:48:33.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38692024-11-19T01:48:33.696ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1139 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 zone_address = fd00:1122:3344:101::2
38702024-11-19T01:48:33.696ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
38712024-11-19T01:48:33.696ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc0ecb0e-e5ab-4a4f-85fb-4da3655d6f2d", "content-length": "26", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
38722024-11-19T01:48:33.696ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1144 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
38732024-11-19T01:48:33.698ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1176 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
38742024-11-19T01:48:33.700ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2024-11-19T01:32:28.835276Z, time_modified: 2024-11-19T01:32:28.835276Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, db_silo: Silo { identity: SiloIdentity { id: 7ae1255b-c149-4b2a-8f4e-7489cdeaaefa, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2024-11-19T01:32:35.963389Z, time_modified: 2024-11-19T01:32:35.963389Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468" }] }) }
38752024-11-19T01:48:33.700ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 171.106449ms iteration = 18 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]}},"default_endpoint":{"silo_id":"7ae1255b-c149-4b2a-8f4e-7489cdeaaefa","tls_certs":["230b491903daebcaf87cf435ebe6f0a0535f8ff306b818acc1839c538f57b468"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
38762024-11-19T01:48:33.716ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
38772024-11-19T01:48:33.716ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
38782024-11-19T01:48:33.716ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38792024-11-19T01:48:33.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38802024-11-19T01:48:33.718ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38812024-11-19T01:48:33.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38822024-11-19T01:48:33.721ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38832024-11-19T01:48:33.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38842024-11-19T01:48:33.723ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 193.380868ms iteration = 37 status = {"physical_disks_added":0}
38852024-11-19T01:48:33.732ZWARNnexus (BackgroundTasks): instance reincarnation completed with errors background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:103 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0 query_errors = 0 restart_errors = 0
38862024-11-19T01:48:33.732ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 200.150258ms iteration = 17 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
38872024-11-19T01:48:33.818ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
38882024-11-19T01:48:33.818ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3033115a-3040-4187-8872-54f13f3dc169", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38892024-11-19T01:48:33.818ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38902024-11-19T01:48:33.821ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "66987141-29e8-41c8-a4b1-4053aa56de0a", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38912024-11-19T01:48:33.821ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38922024-11-19T01:48:33.823ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
38932024-11-19T01:48:33.823ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 293.751833ms iteration = 19 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
38942024-11-19T01:48:33.843ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
38952024-11-19T01:48:33.847ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
38962024-11-19T01:48:33.847ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
38972024-11-19T01:48:33.847ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cffb61a-ab33-4907-aeec-113be622cb15", "content-length": "372", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
38982024-11-19T01:48:33.847ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
38992024-11-19T01:48:33.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39002024-11-19T01:48:33.850ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 5d343227-c6b7-4b08-bad4-2eb0742341f4 dns_zone_name = control-plane.oxide.internal found_so_far = 51 version = 1
39012024-11-19T01:48:33.850ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
39022024-11-19T01:48:33.850ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2024-11-19 01:32:33.664524 UTC dns_group = internal generation = 1
39032024-11-19T01:48:33.850ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 320.320437ms iteration = 18 status = {"generation":1}
39042024-11-19T01:48:33.856ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 326.781983ms iteration = 33 status = {}
39052024-11-19T01:48:33.857ZDEBGnexus (DpdClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen
39062024-11-19T01:48:33.857ZDEBGnexus (DpdClient): client response background_task = nat_v4_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/rpw/nat/ipv4/gen", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1290834-5e96-4f61-a69f-b4b95e370e0a", "content-length": "2", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39072024-11-19T01:48:33.859ZDEBGnexus (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
39082024-11-19T01:48:33.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
39092024-11-19T01:48:33.861ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
39102024-11-19T01:48:33.861ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
39112024-11-19T01:48:33.861ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 331.572269ms iteration = 33 status = null
39122024-11-19T01:48:33.861ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "495e4e86-dd18-4093-8988-8edafb210998", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39132024-11-19T01:48:33.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39142024-11-19T01:48:33.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39152024-11-19T01:48:33.890ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
39162024-11-19T01:48:33.891ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "70d9dac9-cb68-41c6-9258-34b91c1f7fd0", "content-length": "691", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39172024-11-19T01:48:33.891ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
39182024-11-19T01:48:33.891ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
39192024-11-19T01:48:33.891ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
39202024-11-19T01:48:33.892ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 361.857961ms iteration = 34 status = {}
39212024-11-19T01:48:33.893ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39222024-11-19T01:48:33.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39232024-11-19T01:48:33.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39242024-11-19T01:48:33.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39252024-11-19T01:48:33.900ZINFOnexus (BackgroundTasks): VPC route manager sled 302245ee-8785-4baf-86ed-0823410907ec push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
39262024-11-19T01:48:33.900ZDEBGnexus (BackgroundTasks): client request SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
39272024-11-19T01:48:33.900ZDEBGnexus (BackgroundTasks): client response SledAgent = 302245ee-8785-4baf-86ed-0823410907ec background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "61d1b5de-44e2-413b-bcaf-3d956ac50a8f", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39282024-11-19T01:48:33.902ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 370.398803ms iteration = 35 status = {}
39292024-11-19T01:48:33.916ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
39302024-11-19T01:48:33.917ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0f0a572a-eb24-42b9-a5db-e42b210ab0fb", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39312024-11-19T01:48:33.917ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39322024-11-19T01:48:33.918ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "9d19714d-e611-468c-a262-12c60d09564c", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39332024-11-19T01:48:33.918ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39342024-11-19T01:48:33.920ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "38964b44-7b97-44b4-9aa4-7e1ef41284f5", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39352024-11-19T01:48:33.920ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39362024-11-19T01:48:33.921ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
39372024-11-19T01:48:33.921ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 391.446689ms iteration = 19 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
39382024-11-19T01:48:33.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39392024-11-19T01:48:33.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39402024-11-19T01:48:33.962ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:151 port = SwitchPort { id: 2ed5d87d-b512-430b-8a9f-5bee05700135, rack_id: cbab7bfb-8e8e-44f1-a93b-b95de3b50f49, switch_location: "switch0", port_name: "qsfp0", port_settings_id: Some(a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd) } rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
39412024-11-19T01:48:33.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39422024-11-19T01:48:33.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39432024-11-19T01:48:33.971ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron
39442024-11-19T01:48:33.971ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/port/qsfp0/settings?tag=omicron", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f0a42f7-2a09-4a57-a4a8-d400acb5af67", "content-length": "134", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39452024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1746 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = "qsfp0"
39462024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1773 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0 switch_port_id = "qsfp0" switch_port_settings_id = a8b5b6ed-8c4b-4b6d-a2dc-17e991abb0cd
39472024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:383 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
39482024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:387 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
39492024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:396 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39502024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39512024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39522024-11-19T01:48:33.971ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1920 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
39532024-11-19T01:48:33.972ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:413 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
39542024-11-19T01:48:33.972ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
39552024-11-19T01:48:33.972ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:422 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49
39562024-11-19T01:48:33.972ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39572024-11-19T01:48:33.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39582024-11-19T01:48:33.973ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv4
39592024-11-19T01:48:33.974ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv4", status: 200, headers: {"content-type": "application/json", "x-request-id": "065e3b4d-8f35-41f0-822b-b09f29e1160a", "content-length": "2", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39602024-11-19T01:48:33.974ZDEBGnexus (DpdClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::2]:12224/loopback/ipv6
39612024-11-19T01:48:33.974ZDEBGnexus (DpdClient): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12224/loopback/ipv6", status: 200, headers: {"content-type": "application/json", "x-request-id": "89d6b0e2-b4c3-4663-b3a8-d2387fd3c664", "content-length": "51", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39622024-11-19T01:48:33.974ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:471 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch_location = Switch0
39632024-11-19T01:48:33.974ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
39642024-11-19T01:48:34.083ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "32e1142d-e872-471d-9bbe-ca8ba7ff2f16", "date": "Tue, 19 Nov 2024 01:48:33 GMT"} })
39652024-11-19T01:48:34.083ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:774 rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 switch = Switch0
39662024-11-19T01:48:34.083ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
39672024-11-19T01:48:34.087ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = cbab7bfb-8e8e-44f1-a93b-b95de3b50f49 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "01d12e18-8444-4998-ab91-81742f11d4f3", "content-length": "682", "date": "Tue, 19 Nov 2024 01:48:34 GMT"} })
39682024-11-19T01:48:34.088ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39692024-11-19T01:48:34.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39702024-11-19T01:48:34.092ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39712024-11-19T01:48:34.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39722024-11-19T01:48:34.096ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39732024-11-19T01:48:34.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39742024-11-19T01:48:34.098ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39752024-11-19T01:48:34.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39762024-11-19T01:48:34.106ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 575.994759ms iteration = 33 status = {}
39772024-11-19T01:48:34.594ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 920a7646-719b-46b3-a82c-da48d8f2e7d1 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39782024-11-19T01:48:34.594ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 920a7646-719b-46b3-a82c-da48d8f2e7d1 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39792024-11-19T01:48:34.596ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2523 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 920a7646-719b-46b3-a82c-da48d8f2e7d1 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39802024-11-19T01:48:35.619ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 50ab4fdb-8ef9-479e-8f4f-62c47dcb10ef roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39812024-11-19T01:48:35.619ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 50ab4fdb-8ef9-479e-8f4f-62c47dcb10ef resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39822024-11-19T01:48:35.621ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2701 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 50ab4fdb-8ef9-479e-8f4f-62c47dcb10ef response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39832024-11-19T01:48:36.288ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 4e76973e-a14f-4369-8f1c-95625376fe9c roles = RoleSet { roles: {} } uri = /nat/ipv4/changeset/11?limit=100
39842024-11-19T01:48:36.288ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 4e76973e-a14f-4369-8f1c-95625376fe9c resource = Database result = Ok(()) uri = /nat/ipv4/changeset/11?limit=100
39852024-11-19T01:48:36.291ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3015 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::2]:46126 req_id = 4e76973e-a14f-4369-8f1c-95625376fe9c response_code = 200 uri = /nat/ipv4/changeset/11?limit=100
39862024-11-19T01:48:36.643ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 94d14276-9325-428f-9c98-38f40727d90d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39872024-11-19T01:48:36.643ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 94d14276-9325-428f-9c98-38f40727d90d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39882024-11-19T01:48:36.645ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2214 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 94d14276-9325-428f-9c98-38f40727d90d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39892024-11-19T01:48:37.668ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4b83f9ec-2c8f-4af4-9033-7dcb271d69b9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39902024-11-19T01:48:37.669ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4b83f9ec-2c8f-4af4-9033-7dcb271d69b9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39912024-11-19T01:48:37.670ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2265 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 4b83f9ec-2c8f-4af4-9033-7dcb271d69b9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39922024-11-19T01:48:38.694ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6dc50be2-55f7-44d9-b54b-df80d3b13a2b roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39932024-11-19T01:48:38.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6dc50be2-55f7-44d9-b54b-df80d3b13a2b resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39942024-11-19T01:48:38.696ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2190 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6dc50be2-55f7-44d9-b54b-df80d3b13a2b response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
39952024-11-19T01:48:39.163ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::2]:57015
39962024-11-19T01:48:39.163ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:57015 req_id = 6d1c77c5-b28e-4e4b-afb2-a8810fca1535 roles = RoleSet { roles: {} } uri = /metrics/producers
39972024-11-19T01:48:39.163ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:57015 req_id = 6d1c77c5-b28e-4e4b-afb2-a8810fca1535 resource = Database result = Ok(()) uri = /metrics/producers
39982024-11-19T01:48:39.199ZINFOnexus: assigned collector to new producer collector_id = b8403c47-1584-4cc7-8521-f7e7869f6be7 file = nexus/src/app/oximeter.rs:76 producer_id = e775bb0f-d2b5-4c13-b4f1-f5a61d1d3098
39992024-11-19T01:48:39.199ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36294 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::2]:57015 req_id = 6d1c77c5-b28e-4e4b-afb2-a8810fca1535 response_code = 201 uri = /metrics/producers
40002024-11-19T01:48:39.719ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6760b23b-f02b-4ae0-84cb-0d66ab36d0a8 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40012024-11-19T01:48:39.720ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6760b23b-f02b-4ae0-84cb-0d66ab36d0a8 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40022024-11-19T01:48:39.722ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2761 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 6760b23b-f02b-4ae0-84cb-0d66ab36d0a8 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40032024-11-19T01:48:40.745ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd799a2c-f49c-4f51-9d13-43852394f684 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40042024-11-19T01:48:40.745ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd799a2c-f49c-4f51-9d13-43852394f684 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40052024-11-19T01:48:40.747ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2374 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = fd799a2c-f49c-4f51-9d13-43852394f684 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40062024-11-19T01:48:41.771ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a1da7165-3178-4210-8e49-4d9f862d945c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40072024-11-19T01:48:41.771ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a1da7165-3178-4210-8e49-4d9f862d945c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40082024-11-19T01:48:41.772ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1832 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = a1da7165-3178-4210-8e49-4d9f862d945c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40092024-11-19T01:48:41.811ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1726 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 42157a1a-2654-4bd7-8b6e-2529150c0cb9 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73
40102024-11-19T01:48:42.794ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b6cab484-30db-43cb-8636-b9c59b083aa2 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40112024-11-19T01:48:42.795ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b6cab484-30db-43cb-8636-b9c59b083aa2 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40122024-11-19T01:48:42.797ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2337 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = b6cab484-30db-43cb-8636-b9c59b083aa2 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40132024-11-19T01:48:43.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 98 reason = Timeout
40142024-11-19T01:48:43.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40152024-11-19T01:48:43.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40162024-11-19T01:48:43.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40172024-11-19T01:48:43.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 98 reason = Timeout
40182024-11-19T01:48:43.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40192024-11-19T01:48:43.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40202024-11-19T01:48:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40212024-11-19T01:48:43.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40222024-11-19T01:48:43.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40232024-11-19T01:48:43.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40242024-11-19T01:48:43.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40252024-11-19T01:48:43.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40262024-11-19T01:48:43.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
40272024-11-19T01:48:43.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40282024-11-19T01:48:43.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40292024-11-19T01:48:43.551ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
40302024-11-19T01:48:43.551ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 21.317981ms iteration = 98 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
40312024-11-19T01:48:43.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40322024-11-19T01:48:43.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40332024-11-19T01:48:43.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 30.141819ms iteration = 98 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40342024-11-19T01:48:43.819ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 042d2ab6-0e10-43b2-9bf3-7d4ca1bed3c4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40352024-11-19T01:48:43.819ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 042d2ab6-0e10-43b2-9bf3-7d4ca1bed3c4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40362024-11-19T01:48:43.821ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2344 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 042d2ab6-0e10-43b2-9bf3-7d4ca1bed3c4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40372024-11-19T01:48:44.844ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f86e7adf-5698-42b3-bd26-4a3902f609ab roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40382024-11-19T01:48:44.844ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f86e7adf-5698-42b3-bd26-4a3902f609ab resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40392024-11-19T01:48:44.846ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2086 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = f86e7adf-5698-42b3-bd26-4a3902f609ab response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40402024-11-19T01:48:45.868ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0ecb6f0d-50fc-4bb9-9394-af46440e3c8c roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40412024-11-19T01:48:45.868ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0ecb6f0d-50fc-4bb9-9394-af46440e3c8c resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40422024-11-19T01:48:45.870ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2364 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 0ecb6f0d-50fc-4bb9-9394-af46440e3c8c response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40432024-11-19T01:48:46.893ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e4dbd494-96cd-4f9d-99a1-09eef17d419d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40442024-11-19T01:48:46.893ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e4dbd494-96cd-4f9d-99a1-09eef17d419d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40452024-11-19T01:48:46.895ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2239 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e4dbd494-96cd-4f9d-99a1-09eef17d419d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40462024-11-19T01:48:47.917ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 48f357bb-8004-4934-a38c-2e708ea1ce27 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40472024-11-19T01:48:47.917ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 48f357bb-8004-4934-a38c-2e708ea1ce27 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40482024-11-19T01:48:47.919ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2240 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 48f357bb-8004-4934-a38c-2e708ea1ce27 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40492024-11-19T01:48:48.941ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3df775ce-8c98-42de-9a52-46e185baadbe roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40502024-11-19T01:48:48.942ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3df775ce-8c98-42de-9a52-46e185baadbe resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40512024-11-19T01:48:48.944ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2731 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 3df775ce-8c98-42de-9a52-46e185baadbe response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40522024-11-19T01:48:49.966ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 617a2ca0-f885-42bf-ad5d-715d870d774d roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40532024-11-19T01:48:49.967ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 617a2ca0-f885-42bf-ad5d-715d870d774d resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40542024-11-19T01:48:49.969ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2564 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 617a2ca0-f885-42bf-ad5d-715d870d774d response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40552024-11-19T01:48:50.991ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 72de8a07-868e-40b0-ab58-b0e68406f407 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40562024-11-19T01:48:50.991ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 72de8a07-868e-40b0-ab58-b0e68406f407 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40572024-11-19T01:48:50.993ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2138 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 72de8a07-868e-40b0-ab58-b0e68406f407 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40582024-11-19T01:48:52.016ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 45c678e9-c3bb-47f1-958d-50d8d0d882a9 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40592024-11-19T01:48:52.017ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 45c678e9-c3bb-47f1-958d-50d8d0d882a9 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40602024-11-19T01:48:52.019ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2856 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 45c678e9-c3bb-47f1-958d-50d8d0d882a9 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40612024-11-19T01:48:53.041ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 51d811ef-a7f6-4efd-a9eb-f338bd7c26c0 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40622024-11-19T01:48:53.041ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 51d811ef-a7f6-4efd-a9eb-f338bd7c26c0 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40632024-11-19T01:48:53.043ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2584 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 51d811ef-a7f6-4efd-a9eb-f338bd7c26c0 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40642024-11-19T01:48:53.530ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 99 reason = Timeout
40652024-11-19T01:48:53.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40662024-11-19T01:48:53.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40672024-11-19T01:48:53.532ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 99 reason = Timeout
40682024-11-19T01:48:53.532ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40692024-11-19T01:48:53.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40702024-11-19T01:48:53.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40712024-11-19T01:48:53.533ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40722024-11-19T01:48:53.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40732024-11-19T01:48:53.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40742024-11-19T01:48:53.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40752024-11-19T01:48:53.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40762024-11-19T01:48:53.535ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.669246ms iteration = 99 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40772024-11-19T01:48:53.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40782024-11-19T01:48:53.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40792024-11-19T01:48:53.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40802024-11-19T01:48:53.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 42cc80b5-1c54-4919-b680-9adaa2eee120, lookup_type: ById(42cc80b5-1c54-4919-b680-9adaa2eee120) } result = Ok(())
40812024-11-19T01:48:53.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40822024-11-19T01:48:53.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40832024-11-19T01:48:53.549ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 original_time_created = 2024-11-19 01:32:24.851324 UTC target_id = 42cc80b5-1c54-4919-b680-9adaa2eee120 time_created = 2024-11-19 01:32:24.851324 UTC
40842024-11-19T01:48:53.549ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 19.061681ms iteration = 99 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"42cc80b5-1c54-4919-b680-9adaa2eee120","time_created":"2024-11-19T01:32:24.851324Z"}
40852024-11-19T01:48:54.065ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ad633df8-3470-4580-828a-50375639bcf4 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40862024-11-19T01:48:54.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ad633df8-3470-4580-828a-50375639bcf4 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40872024-11-19T01:48:54.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2094 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = ad633df8-3470-4580-828a-50375639bcf4 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40882024-11-19T01:48:55.090ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2c2be3fc-5eb1-496a-a6eb-8ddfe1781a63 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40892024-11-19T01:48:55.090ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2c2be3fc-5eb1-496a-a6eb-8ddfe1781a63 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40902024-11-19T01:48:55.092ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2036 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = 2c2be3fc-5eb1-496a-a6eb-8ddfe1781a63 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40912024-11-19T01:48:56.115ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e196e520-77b5-4c4b-85d2-59cf343ce7a1 roles = RoleSet { roles: {} } uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40922024-11-19T01:48:56.116ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e196e520-77b5-4c4b-85d2-59cf343ce7a1 resource = Database result = Ok(()) uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40932024-11-19T01:48:56.118ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2510 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::1]:55708 req_id = e196e520-77b5-4c4b-85d2-59cf343ce7a1 response_code = 200 uri = /probes/302245ee-8785-4baf-86ed-0823410907ec
40942024-11-19T01:48:56.810ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1625 local_addr = [fd00:1122:3344:101::b]:50058 method = GET remote_addr = [fd00:1122:3344:101::d]:52908 req_id = 2a9d6e5b-0091-434c-aa47-c037b4a47f92 response_code = 200 uri = /13c22fb6-5873-4940-9848-cc9130b7aa73